-- Logs begin at Mon 2022-08-01 13:16:27 UTC, end at Mon 2022-08-01 14:29:34 UTC. -- Aug 01 13:54:36.260722 dv-cinder-847131-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 01 13:54:36.269634 dv-cinder-847131-3-iscsi devstack@keystone.service[80129]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 01 13:54:36.271730 dv-cinder-847131-3-iscsi devstack@keystone.service[80129]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 1 13:54:36 2022] *** Aug 01 13:54:36.271730 dv-cinder-847131-3-iscsi devstack@keystone.service[80129]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 01 13:54:36.271730 dv-cinder-847131-3-iscsi devstack@keystone.service[80129]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 01 13:54:36.271730 dv-cinder-847131-3-iscsi devstack@keystone.service[80129]: nodename: dv-cinder-847131-3-iscsi Aug 01 13:54:36.271730 dv-cinder-847131-3-iscsi devstack@keystone.service[80129]: machine: x86_64 Aug 01 13:54:36.271730 dv-cinder-847131-3-iscsi devstack@keystone.service[80129]: clock source: unix Aug 01 13:54:36.271730 dv-cinder-847131-3-iscsi devstack@keystone.service[80129]: pcre jit disabled Aug 01 13:54:36.271730 dv-cinder-847131-3-iscsi devstack@keystone.service[80129]: detected number of CPU cores: 4 Aug 01 13:54:36.271730 dv-cinder-847131-3-iscsi devstack@keystone.service[80129]: current working directory: / Aug 01 13:54:36.271730 dv-cinder-847131-3-iscsi devstack@keystone.service[80129]: detected binary path: /usr/bin/uwsgi-core Aug 01 13:54:36.271730 dv-cinder-847131-3-iscsi devstack@keystone.service[80129]: your processes number limit is 47680 Aug 01 13:54:36.271730 dv-cinder-847131-3-iscsi devstack@keystone.service[80129]: your memory page size is 4096 bytes Aug 01 13:54:36.271730 dv-cinder-847131-3-iscsi devstack@keystone.service[80129]: detected max file descriptor number: 2048 Aug 01 13:54:36.271730 dv-cinder-847131-3-iscsi devstack@keystone.service[80129]: lock engine: pthread robust mutexes Aug 01 13:54:36.272442 dv-cinder-847131-3-iscsi devstack@keystone.service[80129]: thunder lock: enabled Aug 01 13:54:36.272442 dv-cinder-847131-3-iscsi devstack@keystone.service[80129]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 01 13:54:36.272442 dv-cinder-847131-3-iscsi devstack@keystone.service[80129]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 01 13:54:36.311042 dv-cinder-847131-3-iscsi devstack@keystone.service[80129]: Python main interpreter initialized at 0x56402cbff9c0 Aug 01 13:54:36.311042 dv-cinder-847131-3-iscsi devstack@keystone.service[80129]: python threads support enabled Aug 01 13:54:36.311042 dv-cinder-847131-3-iscsi devstack@keystone.service[80129]: your server socket listen backlog is limited to 100 connections Aug 01 13:54:36.311042 dv-cinder-847131-3-iscsi devstack@keystone.service[80129]: your mercy for graceful operations on workers is 90 seconds Aug 01 13:54:36.311390 dv-cinder-847131-3-iscsi devstack@keystone.service[80129]: mapped 403077 bytes (393 KB) for 2 cores Aug 01 13:54:36.311464 dv-cinder-847131-3-iscsi devstack@keystone.service[80129]: *** Operational MODE: preforking *** Aug 01 13:54:36.311716 dv-cinder-847131-3-iscsi devstack@keystone.service[80129]: *** uWSGI is running in multiple interpreter mode *** Aug 01 13:54:36.311716 dv-cinder-847131-3-iscsi devstack@keystone.service[80129]: spawned uWSGI master process (pid: 80129) Aug 01 13:54:36.311899 dv-cinder-847131-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 01 13:54:36.313640 dv-cinder-847131-3-iscsi devstack@keystone.service[80129]: spawned uWSGI worker 1 (pid: 80140, cores: 1) Aug 01 13:54:36.313640 dv-cinder-847131-3-iscsi devstack@keystone.service[80129]: spawned uWSGI worker 2 (pid: 80141, cores: 1) Aug 01 13:54:36.313640 dv-cinder-847131-3-iscsi devstack@keystone.service[80129]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 01 13:54:41.782054 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 13:54:41.782054 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: from cryptography.utils import int_from_bytes Aug 01 13:54:41.782908 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 13:54:41.782908 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: from cryptography.utils import int_from_bytes Aug 01 13:54:41.788151 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 13:54:41.788151 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: from cryptography.utils import int_from_bytes Aug 01 13:54:41.788702 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 13:54:41.788702 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: from cryptography.utils import int_from_bytes Aug 01 13:54:41.832460 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 01 13:54:41.832664 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 01 13:54:41.832855 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] command line args: [] {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 01 13:54:41.833123 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 01 13:54:41.833298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ================================================================================ {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 01 13:54:41.833551 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] admin_token = **** {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.833794 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] config_dir = [] {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.833979 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.834164 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] config_source = [] {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.834341 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] debug = True {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.834538 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.834927 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.835086 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.835334 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.835583 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.835814 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] list_limit = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.835990 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] log_config_append = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.836356 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.836356 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] log_dir = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.836923 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] log_file = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.836923 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.837556 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 01 13:54:41.837556 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 01 13:54:41.837932 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.838048 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] command line args: [] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 01 13:54:41.838168 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.838297 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 01 13:54:41.838425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.838565 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ================================================================================ {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 01 13:54:41.838682 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.838682 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.838975 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] admin_token = **** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.839155 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.839334 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] config_dir = [] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.839462 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.839640 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.839640 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] config_source = [] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.839888 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.840013 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] debug = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.840147 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.840273 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.840415 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.840614 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.840721 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.840882 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.841009 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.841123 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.841242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.841370 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.841486 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.841618 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] list_limit = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.841618 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] log_config_append = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.841618 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.842000 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.48/identity {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.842129 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] log_dir = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.842301 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] publish_errors = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.842407 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] log_file = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.842563 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.842673 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.842803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.842916 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.843038 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.843166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.843270 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.843379 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.843512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.843629 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.843777 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] standard_threads = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.843897 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.844005 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.844124 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.844239 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.844361 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.844473 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.844613 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.844723 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] use_journal = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.844723 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] use_json = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.844940 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.845056 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] use_stderr = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.845260 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.845377 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] use_syslog = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.845515 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.845628 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.845749 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.845862 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.845968 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.846086 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.846193 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.846358 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.846358 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.846558 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.48/identity {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.846667 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.846810 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] publish_errors = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.846929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.847049 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.847163 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.847271 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.847378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.847499 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.847605 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.847721 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.847848 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.847968 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.847968 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] standard_threads = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.848182 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.848182 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.848389 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.848389 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.848662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.848780 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.848903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.849010 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] use_journal = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.849130 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.849244 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] use_json = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.849244 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] use_stderr = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.849445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.849572 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] use_syslog = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.849677 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.849787 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 13:54:41.849897 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.850006 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.850119 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.850242 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.850346 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.850455 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.850564 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.850673 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.850781 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.850903 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.851029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.851135 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.851244 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.851362 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.851468 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.851662 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.851767 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.851878 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.851997 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.852103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.852221 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.852339 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.852454 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.852584 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.852700 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.852807 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.852915 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.853043 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.853149 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.853149 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.853365 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.853557 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.853712 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.853831 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.853939 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.854049 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.854160 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.854267 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.854376 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.854502 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.854608 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.854717 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.854831 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.854939 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.855060 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.855165 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.855274 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.855382 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.855491 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.855599 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.855720 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.855826 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.855935 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.856045 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.856153 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.856271 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.856394 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.856529 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.856641 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.856751 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.856859 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.857016 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.857016 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.857297 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.857406 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.857516 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.857516 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.857726 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.857726 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.857940 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.858048 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.858158 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.858267 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.858374 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.858495 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.858600 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.858709 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.858818 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.858935 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.859041 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.859166 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.859280 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.859914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.860077 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.860240 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.860392 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.860597 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.860764 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.860914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.861085 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.861240 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.861446 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.861639 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.861759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.861930 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.862037 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.862194 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.862307 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.862475 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.862589 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.862743 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.862862 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.863033 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.863162 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.863162 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.863461 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.863570 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.863749 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.863865 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.864034 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.864034 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.864289 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.864406 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.864593 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.864727 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.864847 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.864964 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.865140 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.865248 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.865424 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.865540 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.865706 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.865814 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.865982 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.866089 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.866244 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.866361 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.866468 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.866600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.866731 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.866838 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.866963 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.867085 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.867195 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.867316 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.867444 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.867551 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.867666 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.867776 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.867887 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.867995 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.868128 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.868237 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.868347 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.868456 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.868602 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.868718 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.868842 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.868949 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.869059 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.869174 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.869284 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.869438 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.869561 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.869669 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.869787 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.869894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.870005 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.870114 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.870304 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.870412 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.870521 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.870632 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.870741 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.870856 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.870978 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.871102 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.871225 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.871332 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.871444 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.871553 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.871663 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.871772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] auth.external = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.871897 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.872003 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.872162 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.872268 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.872381 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.872585 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.872695 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.872806 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] auth.password = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.872916 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.873026 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] auth.token = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.873135 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.873266 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.873374 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.873485 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.873594 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.873705 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.873815 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.873938 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.874050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.874159 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.874333 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.874439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.874551 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.874698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.874816 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.874937 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] credential.caching = True {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.875609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.875718 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.875831 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.875943 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.876054 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.876176 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.876289 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.876400 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.876574 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.876700 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.876810 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.876943 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.877061 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] auth.external = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.877171 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.877285 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.877408 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.877408 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.877632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.877632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.877830 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.877942 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] auth.password = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.878057 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.878170 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] auth.token = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.878295 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.878405 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.878519 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.878634 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.878747 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.878861 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.878986 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.879097 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.879211 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] federation.caching = True {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.879324 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.879445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.879559 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.879686 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.879807 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.879930 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.880042 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] credential.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.880158 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.880271 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.880397 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.880589 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.880714 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.880830 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.880945 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.881064 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.881196 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.881306 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.881420 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.881534 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.881650 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.881764 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.881891 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.882001 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.882116 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] identity.caching = True {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.882230 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.882345 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.882459 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.882586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.882701 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.882815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.882937 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.883117 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.883117 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.883342 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.883451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.883568 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.883681 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.883795 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.883910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.884035 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] federation.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.884146 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.884262 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.884392 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.884537 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.884677 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.884873 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.884983 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.885100 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.885213 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.885328 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.885455 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.885567 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.885926 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.886045 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.886160 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.886282 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.886409 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.886529 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.886642 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.886758 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.886876 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.886989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.887116 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.887281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.887391 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.887508 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.887621 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.887735 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.887862 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.887979 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.888095 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.888228 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.888337 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.888455 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.888609 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.888718 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.888837 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.888953 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.889068 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.889184 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.889313 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.889425 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.889546 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.889660 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.889777 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.889893 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.890019 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.890132 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.890250 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.890367 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.890493 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.890627 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.890755 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.890866 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.890984 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.890984 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.891195 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.891306 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.891435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.891546 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.891660 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.891779 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.891896 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.892022 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.892154 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.892268 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.892386 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.892524 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.892656 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.892771 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.892900 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.893010 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.893127 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.893251 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.893364 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.893480 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.893605 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.893715 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.893831 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.893946 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.894060 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.894173 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.894304 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.894414 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.user = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.894528 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.894643 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.894758 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.894872 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.895002 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.895113 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.895228 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.895350 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.895463 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.895577 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.895703 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.895813 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.895928 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.896056 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.896181 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.896303 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.896415 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.896598 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.896710 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.896825 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.896944 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.897078 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.897199 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.897366 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.897475 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.897597 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.897721 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.897852 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.897968 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.898087 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.898215 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.898331 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.898449 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.898579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.898695 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.898814 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.898931 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.899049 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.899166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.899296 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.899416 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.899533 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.899660 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.899784 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.899902 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.900032 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.900147 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.900266 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.900384 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.900577 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.900687 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.900821 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.900930 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.901045 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.901164 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.901277 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.901391 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.901517 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.901628 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.901743 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.901858 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.901973 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.902086 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.902213 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.902323 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.902437 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.902552 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.902670 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.902782 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.902909 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.903020 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.903134 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.903248 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.903363 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] resource.caching = True {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.903477 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.903603 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.903713 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.903826 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.903941 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.904054 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.904169 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.904301 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.904411 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.904544 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.904663 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.904777 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.904892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.905025 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.905134 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.905250 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.905363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.905478 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.905478 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] role.caching = True {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.905704 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.905814 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] role.driver = sql {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.905935 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.906047 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.906184 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.906302 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.906430 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.906541 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.906656 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.906770 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.906889 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.907002 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.907128 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.907239 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.907352 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.907462 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.907582 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.907696 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.907893 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.908002 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.908118 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.908232 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.908346 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.908461 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.908651 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] resource.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.908761 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.908882 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.909002 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.909121 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.909257 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.909369 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.909488 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.909607 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.909725 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.909843 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.909971 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.910085 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.910204 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.910322 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.910439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.910557 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.910686 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.910804 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.910923 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.911041 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] role.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.911159 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.911277 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] role.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.911407 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.911520 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.911637 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.911753 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.911870 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.911989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.912117 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.912231 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.912352 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.912470 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.912605 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.912723 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.912859 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.912972 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.913089 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.913206 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.913324 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.913442 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.913573 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.913687 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.913805 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.913922 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.914046 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] token.caching = True {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.914162 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.914291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.914406 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.914523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.914640 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.914758 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.914758 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.914981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.915092 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.915211 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.915327 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.915443 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.915575 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.915702 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.915816 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.915933 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.916050 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.916169 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.916298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.916413 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.916603 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.916710 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.916829 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.916948 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.917082 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.917192 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.917312 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.917428 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.917538 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.917654 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.917786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.917901 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.918018 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.918136 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.918262 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] database.connection = **** {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.918390 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.918515 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.918631 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.918773 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.918890 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.919009 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.919126 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.919243 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.919362 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.919491 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.919606 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] token.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.919723 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.919841 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.919959 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.920077 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.920205 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.920321 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.920438 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.920587 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.920705 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.920822 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.920957 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.921071 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.921189 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.921307 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.921425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.921541 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.921672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.922287 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.922412 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.922531 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.922650 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.922650 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.922888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.923000 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.923118 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.923236 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 01 13:54:41.923355 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.923355 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.923355 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.923355 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.923355 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.923355 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.connection = **** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.923355 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.923355 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.923355 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.924330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.924330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.924330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.924330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.924330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.924330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.924330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.924330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.924330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.925273 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.925273 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.925273 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.925273 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.925273 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.925273 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 13:54:41.925273 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 01 13:54:41.947356 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:41.947589 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:41.947815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:41.948873 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:41.949114 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:41.949346 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:41.956191 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:41.957777 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:41.986088 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:41.986322 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:41.987925 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:41.988167 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.039062 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.039330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.089542 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.089778 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.090942 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.091184 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.154965 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.155193 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.155418 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.155598 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.155706 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.155838 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.155922 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.156055 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.156135 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.156306 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.156559 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.156786 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.179634 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.179839 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.180000 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.180173 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.180329 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.180506 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.180666 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.180815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.181031 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.181207 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.181358 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.181507 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.181655 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.181838 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.192599 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.192900 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.205815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.206053 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.206279 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.206463 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.206620 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.206774 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.206928 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.207082 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.207245 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.207403 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.207553 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80140) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.207704 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.207704 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.207704 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 13:54:42.224696 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.224910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 13:54:42.225226 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.225377 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.225528 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 13:54:42.225694 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 13:54:42.225847 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.225997 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.226144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.226291 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 13:54:42.226447 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.226609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.226755 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.226903 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.226903 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.227181 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.236081 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.236352 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.236647 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.236800 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.236950 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.237113 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.237309 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.237594 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.243777 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80140) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:54:42.244573 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:54:42.248538 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80140) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:54:42.248818 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.249099 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.249374 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:54:42.249524 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.249674 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.249830 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.250426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.250640 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.250786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.250933 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.251088 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.251235 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.251389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.251536 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.251685 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.251829 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.251985 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.252142 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.252142 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.252421 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.252593 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.275346 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.276413 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.276860 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80140) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:54:42.277137 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.277953 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:54:42.278235 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.282252 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80140) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:54:42.282531 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.282808 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.283412 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:54:42.283699 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.283981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.289497 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80140) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:54:42.289733 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.290623 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:54:42.290909 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.293864 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80140) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:54:42.294149 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.294439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.294719 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.294994 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.295277 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.295431 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:54:42.295583 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.295734 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.295882 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.296049 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.296200 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.296360 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.296667 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.297548 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.297833 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.312262 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.312591 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.313891 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.314174 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.316500 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80140) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:54:42.318128 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:54:42.319666 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.319959 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.321314 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.321612 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.324329 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80140) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:54:42.326026 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:54:42.327527 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80140) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:54:42.329285 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:54:42.330737 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.331032 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.331318 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.331614 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.331902 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.332194 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.332470 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.332788 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.333075 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.333364 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.333655 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.333934 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.348653 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80140) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:54:42.348934 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.349196 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.349455 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.350560 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:54:42.350846 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.351115 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.351379 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.356311 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.357877 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.358153 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.358292 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.360400 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.360699 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.361565 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80140) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:54:42.361843 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.362130 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.362414 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.362696 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.363557 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:54:42.363850 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.364140 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.364430 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.364742 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.374064 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80140) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:54:42.374342 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.374620 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.374915 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.375197 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.375468 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.375748 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.376202 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:54:42.376512 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.376784 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.377064 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.377346 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.377622 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.377898 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.391614 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80140) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:54:42.393965 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:54:42.394592 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80140) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:54:42.396982 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:54:42.397552 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.399947 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.400142 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.401686 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80140) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:54:42.401971 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.402250 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.402575 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.404071 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:54:42.404362 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.404672 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.409251 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.410848 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80140) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:54:42.411702 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.413365 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:54:42.413795 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.414082 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.414360 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.414628 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.416307 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.416635 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.416931 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.417214 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.423567 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80140) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:54:42.423846 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.425104 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.427346 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:54:42.427645 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.428076 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.435592 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80140) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:54:42.435837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 13:54:42.435975 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.436127 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.436265 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.436412 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.436568 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.436699 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.436829 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.436973 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.437107 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.437239 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.437370 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.437503 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.437643 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.437776 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.437919 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.438132 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.438313 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.438448 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.438579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.438709 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.438834 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.438965 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.439119 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.439201 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80140) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.439325 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.439493 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 13:54:42.472898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56402cbff9c0 pid: 80140 (default app) Aug 01 13:54:42.474375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56402cbff9c0 pid: 80141 (default app) Aug 01 13:54:42.476341 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b2be4-1835-4d15-8f7b-208d37d0598f None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:42.476662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b2be4-1835-4d15-8f7b-208d37d0598f None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:42.476940 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b2be4-1835-4d15-8f7b-208d37d0598f None None] PATH_INFO: `/v3` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:42.481687 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1/1] 10.222.0.48 () {50 vars in 791 bytes} [Mon Aug 1 13:54:42 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:50.129930 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84608500-8967-40e5-b505-59cf6f66ed8e None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:50.130460 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84608500-8967-40e5-b505-59cf6f66ed8e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:50.130460 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84608500-8967-40e5-b505-59cf6f66ed8e None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:50.134875 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1/2] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:54:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:54:50.140777 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e726e-7a21-450d-bf40-e1d93ec8934e None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:50.141197 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e726e-7a21-450d-bf40-e1d93ec8934e None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:50.141331 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e726e-7a21-450d-bf40-e1d93ec8934e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:50.210798 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: INFO dbcounter [None req-996e726e-7a21-450d-bf40-e1d93ec8934e None None] Registered counter for database keystone Aug 01 13:54:50.230298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG oslo_db.sqlalchemy.engines [None req-996e726e-7a21-450d-bf40-e1d93ec8934e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80140) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 01 13:54:50.236537 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writer thread running {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 01 13:54:50.427033 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG passlib.handlers.bcrypt [None req-996e726e-7a21-450d-bf40-e1d93ec8934e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80140) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 01 13:54:50.427460 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG passlib.handlers.bcrypt [None req-996e726e-7a21-450d-bf40-e1d93ec8934e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80140) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 01 13:54:50.481854 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-996e726e-7a21-450d-bf40-e1d93ec8934e None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:50.483500 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-996e726e-7a21-450d-bf40-e1d93ec8934e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:50.615108 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 2/3] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:50 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 476 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 01 13:54:50.697521 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84608500-8967-40e5-b505-59cf6f66ed8e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:50.800683 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: INFO dbcounter [None req-84608500-8967-40e5-b505-59cf6f66ed8e None None] Registered counter for database keystone Aug 01 13:54:50.826400 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG oslo_db.sqlalchemy.engines [None req-84608500-8967-40e5-b505-59cf6f66ed8e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80141) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 01 13:54:50.833123 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writer thread running {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 01 13:54:50.954098 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84608500-8967-40e5-b505-59cf6f66ed8e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:50.958257 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730bef46-9383-4ac4-999b-8d04a91b6739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:50.958892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730bef46-9383-4ac4-999b-8d04a91b6739 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:50.959174 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730bef46-9383-4ac4-999b-8d04a91b6739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:50.959453 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730bef46-9383-4ac4-999b-8d04a91b6739 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:50.967656 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730bef46-9383-4ac4-999b-8d04a91b6739 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:51.020028 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: WARNING keystone.common.rbac_enforcer.enforcer [None req-730bef46-9383-4ac4-999b-8d04a91b6739 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 01 13:54:51.020367 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730bef46-9383-4ac4-999b-8d04a91b6739 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:51.029767 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 2/4] 10.222.0.48 () {58 vars in 1324 bytes} [Mon Aug 1 13:54:50 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 01 13:54:51.035750 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996e726e-7a21-450d-bf40-e1d93ec8934e None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:51.050495 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996e726e-7a21-450d-bf40-e1d93ec8934e None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:51.054851 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c8bea-8f36-4709-91b8-ed0d59d30cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:51.055973 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c8bea-8f36-4709-91b8-ed0d59d30cf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:51.056259 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c8bea-8f36-4709-91b8-ed0d59d30cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:51.056562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c8bea-8f36-4709-91b8-ed0d59d30cf2 admin admin] PATH_INFO: `/v3/projects/b30b7ebe5b2d4e7c8f161e5b78a7a0df` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:51.058066 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c8bea-8f36-4709-91b8-ed0d59d30cf2 admin admin] RBAC: Authorizing `identity:get_project(project_id=b30b7ebe5b2d4e7c8f161e5b78a7a0df)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:51.114111 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a30c8bea-8f36-4709-91b8-ed0d59d30cf2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 01 13:54:51.114456 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c8bea-8f36-4709-91b8-ed0d59d30cf2 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:51.119278 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 3/5] 10.222.0.48 () {56 vars in 1207 bytes} [Mon Aug 1 13:54:51 2022] GET /identity/v3/projects/b30b7ebe5b2d4e7c8f161e5b78a7a0df => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:53.136128 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c84d88-ea0c-455f-a9f8-c31ef7216d99 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:53.136665 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c84d88-ea0c-455f-a9f8-c31ef7216d99 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:53.136665 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c84d88-ea0c-455f-a9f8-c31ef7216d99 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:53.140873 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 3/6] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:54:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:54:53.146224 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d8a61e-aa9e-42c3-a2d2-f9dabf38f779 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:53.146603 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d8a61e-aa9e-42c3-a2d2-f9dabf38f779 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:53.146790 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d8a61e-aa9e-42c3-a2d2-f9dabf38f779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:53.166475 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-79d8a61e-aa9e-42c3-a2d2-f9dabf38f779 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:53.167982 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-79d8a61e-aa9e-42c3-a2d2-f9dabf38f779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:53.176332 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 4/7] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:53 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 01 13:54:53.198285 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c84d88-ea0c-455f-a9f8-c31ef7216d99 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:53.210309 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c84d88-ea0c-455f-a9f8-c31ef7216d99 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:53.214529 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76b90f-21a5-4613-82c4-5a359d853c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:53.215138 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76b90f-21a5-4613-82c4-5a359d853c4b admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:53.215430 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76b90f-21a5-4613-82c4-5a359d853c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:53.215714 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76b90f-21a5-4613-82c4-5a359d853c4b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:53.223374 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76b90f-21a5-4613-82c4-5a359d853c4b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:53.225182 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76b90f-21a5-4613-82c4-5a359d853c4b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:53.234402 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 4/8] 10.222.0.48 () {58 vars in 1324 bytes} [Mon Aug 1 13:54:53 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 13:54:53.240673 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d8a61e-aa9e-42c3-a2d2-f9dabf38f779 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:53.250105 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d8a61e-aa9e-42c3-a2d2-f9dabf38f779 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:53.254439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530b335a-7bfb-44fd-b4b8-11c8324be32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:53.255068 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b335a-7bfb-44fd-b4b8-11c8324be32c admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:53.255356 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b335a-7bfb-44fd-b4b8-11c8324be32c admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:53.255639 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b335a-7bfb-44fd-b4b8-11c8324be32c admin admin] PATH_INFO: `/v3/users/7d5a51326470403490a7155c74035d21` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:53.257050 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530b335a-7bfb-44fd-b4b8-11c8324be32c admin admin] RBAC: Authorizing `identity:get_user(user_id=7d5a51326470403490a7155c74035d21)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:53.258858 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530b335a-7bfb-44fd-b4b8-11c8324be32c admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:53.260677 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 5/9] 10.222.0.48 () {56 vars in 1198 bytes} [Mon Aug 1 13:54:53 2022] GET /identity/v3/users/7d5a51326470403490a7155c74035d21 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:55.358051 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da7197-34ed-4e13-9b75-363d5727b41e None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:55.359109 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da7197-34ed-4e13-9b75-363d5727b41e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:55.359294 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da7197-34ed-4e13-9b75-363d5727b41e None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:55.360674 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 5/10] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:54:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:54:55.365915 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee51907-4b3c-48a2-b53b-4073f6e44ee3 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:55.366326 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee51907-4b3c-48a2-b53b-4073f6e44ee3 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:55.366478 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee51907-4b3c-48a2-b53b-4073f6e44ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:55.385576 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-1ee51907-4b3c-48a2-b53b-4073f6e44ee3 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:55.386754 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-1ee51907-4b3c-48a2-b53b-4073f6e44ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:55.403094 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42273042-197f-42ea-9c5b-f420f6425a2b None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:55.403508 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42273042-197f-42ea-9c5b-f420f6425a2b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:55.403792 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42273042-197f-42ea-9c5b-f420f6425a2b None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:55.405184 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 6/11] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:54:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:54:55.410727 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daccbbb7-9eed-4fb8-a37a-e853021fba7f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:55.411006 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daccbbb7-9eed-4fb8-a37a-e853021fba7f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:55.411278 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daccbbb7-9eed-4fb8-a37a-e853021fba7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:55.453113 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG passlib.handlers.bcrypt [None req-daccbbb7-9eed-4fb8-a37a-e853021fba7f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80141) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 01 13:54:55.453553 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG passlib.handlers.bcrypt [None req-daccbbb7-9eed-4fb8-a37a-e853021fba7f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80141) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 01 13:54:55.468401 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 6/12] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:54:55.489559 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee51907-4b3c-48a2-b53b-4073f6e44ee3 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:55.498829 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-daccbbb7-9eed-4fb8-a37a-e853021fba7f None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:55.500422 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-daccbbb7-9eed-4fb8-a37a-e853021fba7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:55.512542 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:55.591654 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee51907-4b3c-48a2-b53b-4073f6e44ee3 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:55.600763 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 7/13] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:54:55.623558 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daccbbb7-9eed-4fb8-a37a-e853021fba7f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:55.628906 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27274b7-bc25-452e-bbe3-1e1ef826c983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:55.629539 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27274b7-bc25-452e-bbe3-1e1ef826c983 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:55.629828 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27274b7-bc25-452e-bbe3-1e1ef826c983 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:55.630112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27274b7-bc25-452e-bbe3-1e1ef826c983 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:55.696460 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27274b7-bc25-452e-bbe3-1e1ef826c983 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:55.697959 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27274b7-bc25-452e-bbe3-1e1ef826c983 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:55.721826 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:55.722521 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daccbbb7-9eed-4fb8-a37a-e853021fba7f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:55.724787 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:55.760687 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99f1391-6c5d-439c-bee9-67508c1460ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:55.761810 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f1391-6c5d-439c-bee9-67508c1460ce None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:55.762099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f1391-6c5d-439c-bee9-67508c1460ce None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:55.762377 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f1391-6c5d-439c-bee9-67508c1460ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:55.774003 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99f1391-6c5d-439c-bee9-67508c1460ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:54:55.783002 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-f99f1391-6c5d-439c-bee9-67508c1460ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:54:55.783002 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:55.783002 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:55.783002 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:55.783002 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:55.783002 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:55.783002 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:55.783002 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:55.783002 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:55.783002 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:55.783002 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:55.783002 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:55.783002 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:54:55.783002 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:54:55.783002 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:54:55.783002 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:54:55.784099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:54:55.784099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:54:55.784099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:54:55.784099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:54:55.784099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:55.784099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:55.784099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:55.784099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:55.784099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:55.784099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:55.784099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:55.784099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:55.784099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:55.784099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:55.784099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:55.784099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:55.785148 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:55.785148 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:55.785148 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:55.785148 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:55.785148 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:54:55.785148 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:54:55.785148 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:54:55.785148 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:54:55.785148 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:54:55.785148 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:54:55.785148 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:54:55.785148 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:54:55.785148 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 8/14] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 13:54:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:54:55.790135 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99f1391-6c5d-439c-bee9-67508c1460ce None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:55.801660 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 7/15] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:54:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 13:54:55.806916 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27274b7-bc25-452e-bbe3-1e1ef826c983 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:55.895102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99f1391-6c5d-439c-bee9-67508c1460ce None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:55.908902 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27274b7-bc25-452e-bbe3-1e1ef826c983 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:55.927102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151840c6-743c-4be9-a7fa-09747403d450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:55.927721 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151840c6-743c-4be9-a7fa-09747403d450 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:55.928005 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151840c6-743c-4be9-a7fa-09747403d450 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:55.928278 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151840c6-743c-4be9-a7fa-09747403d450 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:55.929025 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151840c6-743c-4be9-a7fa-09747403d450 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:55.929330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151840c6-743c-4be9-a7fa-09747403d450 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:55.930754 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151840c6-743c-4be9-a7fa-09747403d450 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:55.931113 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:55.943514 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 9/16] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:54:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:54:55.943901 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def100f-1a91-4cd3-abf0-9b947ac5e9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:55.944637 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def100f-1a91-4cd3-abf0-9b947ac5e9d5 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:55.944937 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def100f-1a91-4cd3-abf0-9b947ac5e9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:55.945222 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def100f-1a91-4cd3-abf0-9b947ac5e9d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:55.948978 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151840c6-743c-4be9-a7fa-09747403d450 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:55.955881 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def100f-1a91-4cd3-abf0-9b947ac5e9d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:55.957387 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def100f-1a91-4cd3-abf0-9b947ac5e9d5 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:55.975159 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-1def100f-1a91-4cd3-abf0-9b947ac5e9d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:55.975159 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:55.975159 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:55.975159 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:55.975159 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:55.975159 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:55.975159 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:55.975159 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:55.975159 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:55.975159 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:55.975159 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:55.975159 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:55.975159 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:55.975159 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:55.975159 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:55.976140 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:55.976140 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:55.976140 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:55.976140 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:55.976140 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:55.976140 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:55.976140 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:55.976140 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:55.976140 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:55.976140 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:55.976140 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:55.976140 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:55.976140 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:55.976140 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:55.976140 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:55.976140 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:55.976140 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:55.976961 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:55.976961 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:55.976961 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:55.976961 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:54:55.977171 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:55.977237 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 8/17] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:54:55.981956 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def100f-1a91-4cd3-abf0-9b947ac5e9d5 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:56.054401 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151840c6-743c-4be9-a7fa-09747403d450 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:56.090708 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3665a-1146-485d-91b0-aa71110906c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:56.091381 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3665a-1146-485d-91b0-aa71110906c8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:56.091661 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3665a-1146-485d-91b0-aa71110906c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:56.091935 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3665a-1146-485d-91b0-aa71110906c8 None admin] PATH_INFO: `/v3/users/7d5a51326470403490a7155c74035d21` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:56.093675 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b3665a-1146-485d-91b0-aa71110906c8 None admin] RBAC: Authorizing `identity:get_user(user_id=7d5a51326470403490a7155c74035d21)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:56.095269 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b3665a-1146-485d-91b0-aa71110906c8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:56.097129 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 10/18] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:54:55 2022] GET /identity/v3/users/7d5a51326470403490a7155c74035d21 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:54:56.102810 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3665a-1146-485d-91b0-aa71110906c8 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:56.190065 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def100f-1a91-4cd3-abf0-9b947ac5e9d5 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:56.201659 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:56.209001 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3665a-1146-485d-91b0-aa71110906c8 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:56.223584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ecf6c8-7a4f-4777-ac78-ce1f134bed15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:56.224214 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ecf6c8-7a4f-4777-ac78-ce1f134bed15 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:56.224529 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ecf6c8-7a4f-4777-ac78-ce1f134bed15 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:56.224817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ecf6c8-7a4f-4777-ac78-ce1f134bed15 None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:56.225520 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ecf6c8-7a4f-4777-ac78-ce1f134bed15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:56.225827 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ecf6c8-7a4f-4777-ac78-ce1f134bed15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:56.227221 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ecf6c8-7a4f-4777-ac78-ce1f134bed15 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:56.242137 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb41fb-6d94-4237-aec9-727012b605a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:56.242786 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb41fb-6d94-4237-aec9-727012b605a6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:56.243073 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb41fb-6d94-4237-aec9-727012b605a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:56.243352 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb41fb-6d94-4237-aec9-727012b605a6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:56.244536 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fb41fb-6d94-4237-aec9-727012b605a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:56.244971 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 9/19] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:54:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:56.245989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fb41fb-6d94-4237-aec9-727012b605a6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:56.252719 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 11/20] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:54:56.254172 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ecf6c8-7a4f-4777-ac78-ce1f134bed15 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:56.290387 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:56.361100 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ecf6c8-7a4f-4777-ac78-ce1f134bed15 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:56.396512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eddbbf8-0e62-4ebb-bbad-80891b491ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:56.397161 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eddbbf8-0e62-4ebb-bbad-80891b491ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:56.397448 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eddbbf8-0e62-4ebb-bbad-80891b491ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:56.397731 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eddbbf8-0e62-4ebb-bbad-80891b491ae8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:56.398534 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eddbbf8-0e62-4ebb-bbad-80891b491ae8 None admin] RBAC: Adding query filter params (role.id=58f91e3a13664dabb18c6227a1658fa6, scope.domain.id=default, user.id=7d5a51326470403490a7155c74035d21) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:56.398850 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eddbbf8-0e62-4ebb-bbad-80891b491ae8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:56.400373 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eddbbf8-0e62-4ebb-bbad-80891b491ae8 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:56.407920 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 10/21] 10.222.0.48 () {56 vars in 1322 bytes} [Mon Aug 1 13:54:56 2022] GET /identity/v3/role_assignments?user.id=7d5a51326470403490a7155c74035d21&scope.domain.id=default&role.id=58f91e3a13664dabb18c6227a1658fa6 => generated 233 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:58.814542 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2224b70-8ec6-4577-a997-d6fcf7ddccc2 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:58.815550 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2224b70-8ec6-4577-a997-d6fcf7ddccc2 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:58.816038 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2224b70-8ec6-4577-a997-d6fcf7ddccc2 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:58.818208 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 12/22] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:54:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:54:58.823735 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38533ea9-7eed-420b-b7e8-6d68d321fc20 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:58.824635 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38533ea9-7eed-420b-b7e8-6d68d321fc20 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:58.825123 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38533ea9-7eed-420b-b7e8-6d68d321fc20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:58.853642 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-38533ea9-7eed-420b-b7e8-6d68d321fc20 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:58.855246 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-38533ea9-7eed-420b-b7e8-6d68d321fc20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:58.947068 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 11/23] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:54:58.983507 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2224b70-8ec6-4577-a997-d6fcf7ddccc2 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:59.043545 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24500b8-507a-4f6c-913f-edab02190959 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:59.044171 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24500b8-507a-4f6c-913f-edab02190959 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:59.044715 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24500b8-507a-4f6c-913f-edab02190959 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:59.046850 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 12/24] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:54:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:54:59.052264 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e99a9d-82eb-4df8-b5ba-78a27c9a3c88 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:59.052798 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e99a9d-82eb-4df8-b5ba-78a27c9a3c88 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:59.053280 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e99a9d-82eb-4df8-b5ba-78a27c9a3c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:59.080218 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-c6e99a9d-82eb-4df8-b5ba-78a27c9a3c88 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:59.081369 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-c6e99a9d-82eb-4df8-b5ba-78a27c9a3c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:59.127176 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:59.186213 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2224b70-8ec6-4577-a997-d6fcf7ddccc2 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:59.219382 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 13/25] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:54:59.220977 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b2146-7c75-40af-b4f3-7bba6b637953 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:59.221441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b2146-7c75-40af-b4f3-7bba6b637953 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:59.221875 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b2146-7c75-40af-b4f3-7bba6b637953 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:59.223421 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 14/26] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:54:59.224461 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244cb5f3-40c0-49f9-abab-5238fac96ba7 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:59.228600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244cb5f3-40c0-49f9-abab-5238fac96ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:59.228890 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244cb5f3-40c0-49f9-abab-5238fac96ba7 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:59.230733 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 15/27] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:54:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:54:59.232035 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e7a69-cbc8-4bae-bc88-b7d7f961b429 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:59.232782 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e7a69-cbc8-4bae-bc88-b7d7f961b429 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:59.233234 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e7a69-cbc8-4bae-bc88-b7d7f961b429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:59.234320 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02108e24-68f7-4ceb-a775-3a5679d2b386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:59.235211 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02108e24-68f7-4ceb-a775-3a5679d2b386 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:59.235696 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02108e24-68f7-4ceb-a775-3a5679d2b386 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:59.236295 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02108e24-68f7-4ceb-a775-3a5679d2b386 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:59.249882 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02108e24-68f7-4ceb-a775-3a5679d2b386 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:54:59.251762 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-eb3e7a69-cbc8-4bae-bc88-b7d7f961b429 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:59.253172 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-eb3e7a69-cbc8-4bae-bc88-b7d7f961b429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:59.257049 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-02108e24-68f7-4ceb-a775-3a5679d2b386 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:54:59.257049 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:59.257049 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:59.257049 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:59.257049 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:59.257049 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:59.257049 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:59.257049 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:59.257049 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:59.257049 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:59.257049 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:59.257049 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:59.257049 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:54:59.257049 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:54:59.257049 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:54:59.257049 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:54:59.257870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:54:59.257870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:54:59.257870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:54:59.257870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:54:59.257870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:59.257870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:59.257870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:59.257870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:59.257870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:59.257870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:59.257870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:59.257870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:59.257870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:59.257870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:59.257870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:59.257870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:59.258608 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:59.258608 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:59.258608 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:59.258608 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:59.258608 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:54:59.258608 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:54:59.258608 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:54:59.258608 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:54:59.258608 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:54:59.258608 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:54:59.258608 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:54:59.258608 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:54:59.261134 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 13/28] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 13:54:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:54:59.262554 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace02598-b2df-495a-9f31-08383b865974 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:59.263025 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace02598-b2df-495a-9f31-08383b865974 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:59.263445 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace02598-b2df-495a-9f31-08383b865974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:59.290746 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ace02598-b2df-495a-9f31-08383b865974 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:54:59.292953 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ace02598-b2df-495a-9f31-08383b865974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:54:59.315023 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:59.333902 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 16/29] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:54:59.339801 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e7a69-cbc8-4bae-bc88-b7d7f961b429 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:59.379941 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 14/30] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:54:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:54:59.381527 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace02598-b2df-495a-9f31-08383b865974 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:59.463521 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:59.477776 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e7a69-cbc8-4bae-bc88-b7d7f961b429 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:59.511352 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace02598-b2df-495a-9f31-08383b865974 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:59.522866 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a3c38-814c-4bde-b029-04cac949ecc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:59.524045 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a3c38-814c-4bde-b029-04cac949ecc7 None admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:59.524751 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a3c38-814c-4bde-b029-04cac949ecc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:59.525131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a3c38-814c-4bde-b029-04cac949ecc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:59.526192 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a3c38-814c-4bde-b029-04cac949ecc7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:59.528372 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a3c38-814c-4bde-b029-04cac949ecc7 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:59.551132 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b653c5-6106-427f-932d-88ad745fd4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:59.551763 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b653c5-6106-427f-932d-88ad745fd4a5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:59.552092 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b653c5-6106-427f-932d-88ad745fd4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:59.552377 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b653c5-6106-427f-932d-88ad745fd4a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:59.553136 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b653c5-6106-427f-932d-88ad745fd4a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:54:59.553443 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b653c5-6106-427f-932d-88ad745fd4a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:59.554866 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b653c5-6106-427f-932d-88ad745fd4a5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:59.569258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 15/31] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:54:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:54:59.570187 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b653c5-6106-427f-932d-88ad745fd4a5 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:59.588464 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:59.663340 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 17/32] 10.222.0.48 () {60 vars in 1131 bytes} [Mon Aug 1 13:54:59 2022] POST /identity/v3/roles => generated 225 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:54:59.664132 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a3c38-814c-4bde-b029-04cac949ecc7 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:59.708552 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b653c5-6106-427f-932d-88ad745fd4a5 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:59.776056 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a3c38-814c-4bde-b029-04cac949ecc7 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:54:59.784857 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80fb1a3-0c9c-4fd6-a94c-16dadbcf7135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:59.785519 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80fb1a3-0c9c-4fd6-a94c-16dadbcf7135 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:59.785786 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80fb1a3-0c9c-4fd6-a94c-16dadbcf7135 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:59.786059 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80fb1a3-0c9c-4fd6-a94c-16dadbcf7135 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:59.815265 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424c8c1-ae47-41d4-a890-676f5296899a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:54:59.815921 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424c8c1-ae47-41d4-a890-676f5296899a None admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:54:59.816207 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424c8c1-ae47-41d4-a890-676f5296899a None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:54:59.816502 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424c8c1-ae47-41d4-a890-676f5296899a None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:54:59.817660 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a424c8c1-ae47-41d4-a890-676f5296899a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:59.818804 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a424c8c1-ae47-41d4-a890-676f5296899a None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:59.854765 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 18/33] 10.222.0.48 () {60 vars in 1131 bytes} [Mon Aug 1 13:54:59 2022] POST /identity/v3/roles => generated 219 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:54:59.855630 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424c8c1-ae47-41d4-a890-676f5296899a None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:54:59.876994 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:54:59.971008 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80fb1a3-0c9c-4fd6-a94c-16dadbcf7135 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:54:59.973104 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80fb1a3-0c9c-4fd6-a94c-16dadbcf7135 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:54:59.994156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-c80fb1a3-0c9c-4fd6-a94c-16dadbcf7135 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:59.994156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:54:59.994156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:54:59.994156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:54:59.994156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:54:59.994156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:54:59.994156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:54:59.994156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:54:59.994156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:54:59.994156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:54:59.994156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:54:59.994156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:54:59.994156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:54:59.994156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:54:59.994156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:54:59.995161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:54:59.995161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:54:59.995161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:54:59.995161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:54:59.995161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:54:59.995161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:54:59.995161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:54:59.995161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:54:59.995161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:54:59.995161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:54:59.995161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:54:59.995161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:54:59.995161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:54:59.995161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:54:59.995161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:54:59.995161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:54:59.995161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:54:59.995939 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:54:59.995939 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:54:59.995939 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:54:59.995939 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:54:59.998467 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 16/34] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:54:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 429 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:00.006325 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f50017-84a8-4ba4-b0c8-075ea50c6cef None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:00.006631 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f50017-84a8-4ba4-b0c8-075ea50c6cef None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:00.006900 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f50017-84a8-4ba4-b0c8-075ea50c6cef None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:00.008462 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 17/35] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:00.009121 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f50017-84a8-4ba4-b0c8-075ea50c6cef None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:00.056279 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424c8c1-ae47-41d4-a890-676f5296899a None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:00.099910 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:00.101914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b80a5b-24cd-4b0f-af26-a068e055bb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:00.102713 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b80a5b-24cd-4b0f-af26-a068e055bb75 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:00.103008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b80a5b-24cd-4b0f-af26-a068e055bb75 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:00.103290 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b80a5b-24cd-4b0f-af26-a068e055bb75 None admin] PATH_INFO: `/v3/users/7d5a51326470403490a7155c74035d21` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:00.104608 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b80a5b-24cd-4b0f-af26-a068e055bb75 None admin] RBAC: Authorizing `identity:get_user(user_id=7d5a51326470403490a7155c74035d21)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:00.106130 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b80a5b-24cd-4b0f-af26-a068e055bb75 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:00.108398 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 19/36] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:54:59 2022] GET /identity/v3/users/7d5a51326470403490a7155c74035d21 => generated 247 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:00.112787 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a67b4-62c8-4ee0-b343-679ed9f3d985 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:00.112787 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a67b4-62c8-4ee0-b343-679ed9f3d985 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:00.112787 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a67b4-62c8-4ee0-b343-679ed9f3d985 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:00.112787 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 20/37] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:00.112787 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb90f5-dc89-4f92-8fcd-d752ae0ae602 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:00.116622 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb90f5-dc89-4f92-8fcd-d752ae0ae602 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:00.117173 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb90f5-dc89-4f92-8fcd-d752ae0ae602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:00.137803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-3afb90f5-dc89-4f92-8fcd-d752ae0ae602 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:00.139414 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-3afb90f5-dc89-4f92-8fcd-d752ae0ae602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:00.142830 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f50017-84a8-4ba4-b0c8-075ea50c6cef None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:00.177045 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a99df3-b3c8-468e-b2f2-3681754a0e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:00.177664 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a99df3-b3c8-468e-b2f2-3681754a0e26 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:00.177946 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a99df3-b3c8-468e-b2f2-3681754a0e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:00.178223 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a99df3-b3c8-468e-b2f2-3681754a0e26 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:00.178918 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a99df3-b3c8-468e-b2f2-3681754a0e26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:00.179221 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a99df3-b3c8-468e-b2f2-3681754a0e26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:00.180619 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a99df3-b3c8-468e-b2f2-3681754a0e26 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:00.198146 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 18/38] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:55:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:00.198767 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a99df3-b3c8-468e-b2f2-3681754a0e26 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:00.206556 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 21/39] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:00.210720 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0db7a6-edab-409b-b906-22408eeab1af None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:00.210720 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0db7a6-edab-409b-b906-22408eeab1af None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:00.210720 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0db7a6-edab-409b-b906-22408eeab1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:00.223965 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:00.231350 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-be0db7a6-edab-409b-b906-22408eeab1af None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:00.232564 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-be0db7a6-edab-409b-b906-22408eeab1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:00.300950 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 22/40] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:00.301690 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0db7a6-edab-409b-b906-22408eeab1af None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:00.311184 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a99df3-b3c8-468e-b2f2-3681754a0e26 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:00.337827 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:00.347671 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd76200-2f22-430a-8e4b-3e4a539a2938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:00.348327 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd76200-2f22-430a-8e4b-3e4a539a2938 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:00.348632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd76200-2f22-430a-8e4b-3e4a539a2938 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:00.348908 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd76200-2f22-430a-8e4b-3e4a539a2938 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:00.350068 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd76200-2f22-430a-8e4b-3e4a539a2938 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:00.351505 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd76200-2f22-430a-8e4b-3e4a539a2938 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:00.353330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 19/41] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:55:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:00.353883 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd76200-2f22-430a-8e4b-3e4a539a2938 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:00.403025 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0db7a6-edab-409b-b906-22408eeab1af None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:00.438617 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fb92a-2b10-4d10-a489-8df7badf0493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:00.439247 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fb92a-2b10-4d10-a489-8df7badf0493 None admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:00.439540 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fb92a-2b10-4d10-a489-8df7badf0493 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:00.439823 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fb92a-2b10-4d10-a489-8df7badf0493 None admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:00.440533 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2fb92a-2b10-4d10-a489-8df7badf0493 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:00.441990 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2fb92a-2b10-4d10-a489-8df7badf0493 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:00.448349 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:00.456861 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd76200-2f22-430a-8e4b-3e4a539a2938 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:00.472242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 23/42] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 13:55:00 2022] POST /identity/v3/projects => generated 301 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:55:00.473178 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fb92a-2b10-4d10-a489-8df7badf0493 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:00.508469 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d9b78f-b91c-49cc-9adb-47aa7642cef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:00.509518 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9b78f-b91c-49cc-9adb-47aa7642cef6 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:00.510267 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9b78f-b91c-49cc-9adb-47aa7642cef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:00.510549 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9b78f-b91c-49cc-9adb-47aa7642cef6 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:00.511293 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d9b78f-b91c-49cc-9adb-47aa7642cef6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:00.512740 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d9b78f-b91c-49cc-9adb-47aa7642cef6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:00.587599 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fb92a-2b10-4d10-a489-8df7badf0493 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:00.627532 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ef0266-1dfa-4589-ad75-5aaa9f22453a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:00.628198 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef0266-1dfa-4589-ad75-5aaa9f22453a None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:00.628514 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef0266-1dfa-4589-ad75-5aaa9f22453a None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:00.628882 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef0266-1dfa-4589-ad75-5aaa9f22453a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:00.630068 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ef0266-1dfa-4589-ad75-5aaa9f22453a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:00.631537 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ef0266-1dfa-4589-ad75-5aaa9f22453a None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:00.633235 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 24/43] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:55:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:00.634363 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ef0266-1dfa-4589-ad75-5aaa9f22453a None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:00.641178 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 20/44] 10.222.0.48 () {60 vars in 1131 bytes} [Mon Aug 1 13:55:00 2022] POST /identity/v3/roles => generated 223 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:55:00.642010 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d9b78f-b91c-49cc-9adb-47aa7642cef6 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:00.713442 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:00.728002 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:00.768224 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ef0266-1dfa-4589-ad75-5aaa9f22453a None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:00.782584 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d9b78f-b91c-49cc-9adb-47aa7642cef6 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:00.806712 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a00f4-755b-4c3e-abe9-6c70b1660c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:00.807417 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a00f4-755b-4c3e-abe9-6c70b1660c94 None admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:00.807708 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a00f4-755b-4c3e-abe9-6c70b1660c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:00.807991 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a00f4-755b-4c3e-abe9-6c70b1660c94 None admin] PATH_INFO: `/v3/domains/default/users/7d5a51326470403490a7155c74035d21/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:00.809975 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a00f4-755b-4c3e-abe9-6c70b1660c94 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=7d5a51326470403490a7155c74035d21, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:00.811529 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a00f4-755b-4c3e-abe9-6c70b1660c94 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:00.828839 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05265d2-76d5-436c-870c-382f9e05585a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:00.829482 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05265d2-76d5-436c-870c-382f9e05585a None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:00.829864 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 25/45] 10.222.0.48 () {58 vars in 1361 bytes} [Mon Aug 1 13:55:00 2022] PUT /identity/v3/domains/default/users/7d5a51326470403490a7155c74035d21/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:55:00.829995 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05265d2-76d5-436c-870c-382f9e05585a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:00.830083 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05265d2-76d5-436c-870c-382f9e05585a None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:00.830740 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05265d2-76d5-436c-870c-382f9e05585a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:00.832332 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05265d2-76d5-436c-870c-382f9e05585a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:00.867080 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 21/46] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 13:55:00 2022] POST /identity/v3/projects => generated 312 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:55:02.848771 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fffddf-07bb-4413-8148-92930962a94d None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:02.849210 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fffddf-07bb-4413-8148-92930962a94d None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:02.849348 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fffddf-07bb-4413-8148-92930962a94d None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:02.850728 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 26/47] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:02.856089 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b8df2-11e1-4922-848f-6f91ffa2c164 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:02.856501 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b8df2-11e1-4922-848f-6f91ffa2c164 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:02.856689 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b8df2-11e1-4922-848f-6f91ffa2c164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:02.866316 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e2582-f69a-4249-8bb6-b79f58565ff4 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:02.866601 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e2582-f69a-4249-8bb6-b79f58565ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:02.866882 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e2582-f69a-4249-8bb6-b79f58565ff4 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:02.868181 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 27/48] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:55:02.873133 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09883d5d-4563-48e6-ade0-388822b1348b None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:02.873416 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09883d5d-4563-48e6-ade0-388822b1348b None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:02.873692 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09883d5d-4563-48e6-ade0-388822b1348b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:02.875491 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-fa5b8df2-11e1-4922-848f-6f91ffa2c164 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:02.877043 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fa5b8df2-11e1-4922-848f-6f91ffa2c164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:02.891568 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-09883d5d-4563-48e6-ade0-388822b1348b None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:02.892757 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-09883d5d-4563-48e6-ade0-388822b1348b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:02.945985 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 22/49] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:02.959034 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 28/50] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:55:02.969169 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b8df2-11e1-4922-848f-6f91ffa2c164 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:02.981275 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09883d5d-4563-48e6-ade0-388822b1348b None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:03.002618 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:03.027457 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:03.067579 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b8df2-11e1-4922-848f-6f91ffa2c164 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:03.080945 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09883d5d-4563-48e6-ade0-388822b1348b None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:03.100983 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503678a-2d88-400d-a029-65041dbfad52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:03.101643 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503678a-2d88-400d-a029-65041dbfad52 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:03.101932 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503678a-2d88-400d-a029-65041dbfad52 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:03.102214 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503678a-2d88-400d-a029-65041dbfad52 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:03.112139 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503678a-2d88-400d-a029-65041dbfad52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:03.113511 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e754402-c2b1-4fe1-ade1-af9f8a6c70dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:03.114151 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e754402-c2b1-4fe1-ade1-af9f8a6c70dd None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:03.114436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e754402-c2b1-4fe1-ade1-af9f8a6c70dd None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:03.114714 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e754402-c2b1-4fe1-ade1-af9f8a6c70dd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:03.115867 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e754402-c2b1-4fe1-ade1-af9f8a6c70dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:03.117133 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-1503678a-2d88-400d-a029-65041dbfad52 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:55:03.117133 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:03.117133 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:03.117133 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:03.117133 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:03.117133 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:03.117133 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:03.117133 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:03.117133 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:03.117133 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:03.117133 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:03.117133 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:03.117133 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:03.117133 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:03.117133 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:03.117133 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:03.118099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:03.118099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:03.118099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:03.118099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:03.118099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:03.118099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:03.118099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:03.118099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:03.118099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:03.118099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:03.118099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:03.118099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:03.118099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:03.118099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:03.118099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:03.118099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:03.118870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:03.118870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:03.118870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:03.118870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:03.118870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:03.118870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:03.118870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:03.118870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:03.118870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:03.118870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:03.118870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:55:03.118870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:55:03.119430 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e754402-c2b1-4fe1-ade1-af9f8a6c70dd None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:03.119493 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 23/51] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 13:55:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:03.119563 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 29/52] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:55:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:03.124196 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503678a-2d88-400d-a029-65041dbfad52 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:03.124306 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e754402-c2b1-4fe1-ade1-af9f8a6c70dd None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:03.218727 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:03.223189 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e754402-c2b1-4fe1-ade1-af9f8a6c70dd None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:03.224930 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503678a-2d88-400d-a029-65041dbfad52 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:03.236904 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:03.257569 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32afe21c-8e80-48a3-b023-ff73d538d57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:03.258196 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32afe21c-8e80-48a3-b023-ff73d538d57d None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:03.258489 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32afe21c-8e80-48a3-b023-ff73d538d57d None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:03.258773 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32afe21c-8e80-48a3-b023-ff73d538d57d None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:03.258887 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c181c-6b89-4da1-9db1-94be78dc7cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:03.259463 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c181c-6b89-4da1-9db1-94be78dc7cac None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:03.259550 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32afe21c-8e80-48a3-b023-ff73d538d57d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:03.260002 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c181c-6b89-4da1-9db1-94be78dc7cac None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:03.260093 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32afe21c-8e80-48a3-b023-ff73d538d57d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:03.260278 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c181c-6b89-4da1-9db1-94be78dc7cac None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:03.260989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455c181c-6b89-4da1-9db1-94be78dc7cac None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:03.261456 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32afe21c-8e80-48a3-b023-ff73d538d57d None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:03.262419 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455c181c-6b89-4da1-9db1-94be78dc7cac None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:03.273539 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 30/53] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:03.279261 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32afe21c-8e80-48a3-b023-ff73d538d57d None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:03.290046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 24/54] 10.222.0.48 () {60 vars in 1140 bytes} [Mon Aug 1 13:55:03 2022] POST /identity/v3/projects => generated 298 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:55:03.401113 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32afe21c-8e80-48a3-b023-ff73d538d57d None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:03.436366 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99414dde-8c6f-42d4-b86f-9262080ddbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:03.437057 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99414dde-8c6f-42d4-b86f-9262080ddbcc None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:03.437340 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99414dde-8c6f-42d4-b86f-9262080ddbcc None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:03.437624 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99414dde-8c6f-42d4-b86f-9262080ddbcc None admin] PATH_INFO: `/v3/users/7d5a51326470403490a7155c74035d21` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:03.438840 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99414dde-8c6f-42d4-b86f-9262080ddbcc None admin] RBAC: Authorizing `identity:get_user(user_id=7d5a51326470403490a7155c74035d21)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:03.440311 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99414dde-8c6f-42d4-b86f-9262080ddbcc None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:03.442013 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 31/55] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:03 2022] GET /identity/v3/users/7d5a51326470403490a7155c74035d21 => generated 247 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:03.447633 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c181c-6b89-4da1-9db1-94be78dc7cac None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:03.548725 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c181c-6b89-4da1-9db1-94be78dc7cac None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:03.582126 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815cdaf-f303-47d3-9edf-ef1af793b5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:03.582789 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815cdaf-f303-47d3-9edf-ef1af793b5d0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:03.583079 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815cdaf-f303-47d3-9edf-ef1af793b5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:03.583363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815cdaf-f303-47d3-9edf-ef1af793b5d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:03.584547 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7815cdaf-f303-47d3-9edf-ef1af793b5d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:03.586008 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7815cdaf-f303-47d3-9edf-ef1af793b5d0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:03.587614 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 25/56] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:55:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:03.593433 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99414dde-8c6f-42d4-b86f-9262080ddbcc None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:03.639553 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:03.693649 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99414dde-8c6f-42d4-b86f-9262080ddbcc None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:03.727366 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1672565d-617d-42e3-9da1-426a290e020f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:03.728025 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672565d-617d-42e3-9da1-426a290e020f None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:03.728460 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672565d-617d-42e3-9da1-426a290e020f None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:03.728776 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672565d-617d-42e3-9da1-426a290e020f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:03.729615 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1672565d-617d-42e3-9da1-426a290e020f None admin] RBAC: Adding query filter params (role.id=58f91e3a13664dabb18c6227a1658fa6, scope.domain.id=default, user.id=7d5a51326470403490a7155c74035d21) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:03.729936 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1672565d-617d-42e3-9da1-426a290e020f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:03.731447 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1672565d-617d-42e3-9da1-426a290e020f None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:03.738641 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 32/57] 10.222.0.48 () {56 vars in 1322 bytes} [Mon Aug 1 13:55:03 2022] GET /identity/v3/role_assignments?user.id=7d5a51326470403490a7155c74035d21&scope.domain.id=default&role.id=58f91e3a13664dabb18c6227a1658fa6 => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:05.274583 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb3fb46-c39f-4d73-83f3-0630dff68388 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:05.275091 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb3fb46-c39f-4d73-83f3-0630dff68388 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:05.275173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb3fb46-c39f-4d73-83f3-0630dff68388 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:05.276538 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 26/58] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:05.281650 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5380cd1-2bb4-45d3-a29b-1413c21887a4 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:05.282029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5380cd1-2bb4-45d3-a29b-1413c21887a4 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:05.282229 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5380cd1-2bb4-45d3-a29b-1413c21887a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:05.300399 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-b5380cd1-2bb4-45d3-a29b-1413c21887a4 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:05.301637 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-b5380cd1-2bb4-45d3-a29b-1413c21887a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:05.371439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 33/59] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:05.397299 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb3fb46-c39f-4d73-83f3-0630dff68388 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:05.415971 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:05.498432 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb3fb46-c39f-4d73-83f3-0630dff68388 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:05.530994 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2082635-cfb9-44ad-9dd9-f3026fcce60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:05.531580 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2082635-cfb9-44ad-9dd9-f3026fcce60d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:05.531867 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2082635-cfb9-44ad-9dd9-f3026fcce60d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:05.532147 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2082635-cfb9-44ad-9dd9-f3026fcce60d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:05.533326 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2082635-cfb9-44ad-9dd9-f3026fcce60d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:05.534780 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2082635-cfb9-44ad-9dd9-f3026fcce60d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:05.536350 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 27/60] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:55:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:05.542033 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5380cd1-2bb4-45d3-a29b-1413c21887a4 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:05.580882 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:05.640272 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5380cd1-2bb4-45d3-a29b-1413c21887a4 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:05.672110 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb144654-622b-49a2-b02c-6be8f29f8c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:05.672752 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb144654-622b-49a2-b02c-6be8f29f8c13 None admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:05.673037 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb144654-622b-49a2-b02c-6be8f29f8c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:05.673315 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb144654-622b-49a2-b02c-6be8f29f8c13 None admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:05.674001 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb144654-622b-49a2-b02c-6be8f29f8c13 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:05.675440 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb144654-622b-49a2-b02c-6be8f29f8c13 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:05.702007 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 34/61] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 13:55:05 2022] POST /identity/v3/users => generated 275 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:55:08.768582 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b18084-43e2-469b-a858-9ff52ce0f7c6 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:08.769117 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b18084-43e2-469b-a858-9ff52ce0f7c6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:08.769208 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b18084-43e2-469b-a858-9ff52ce0f7c6 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:08.770647 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 28/62] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:08.775671 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f33f41-9df9-40ea-8920-f6705c4a2c57 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:08.776058 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f33f41-9df9-40ea-8920-f6705c4a2c57 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:08.776238 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f33f41-9df9-40ea-8920-f6705c4a2c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:08.800061 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-e4f33f41-9df9-40ea-8920-f6705c4a2c57 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:08.801322 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-e4f33f41-9df9-40ea-8920-f6705c4a2c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:08.840691 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259dbd16-b950-489a-ba27-404532c008f7 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:08.840989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259dbd16-b950-489a-ba27-404532c008f7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:08.841277 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259dbd16-b950-489a-ba27-404532c008f7 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:08.842675 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 29/63] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:08.851038 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a34315-ad94-4152-803f-0379dbed0750 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:08.851333 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a34315-ad94-4152-803f-0379dbed0750 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:08.851625 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a34315-ad94-4152-803f-0379dbed0750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:08.880416 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b3a34315-ad94-4152-803f-0379dbed0750 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:08.881663 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b3a34315-ad94-4152-803f-0379dbed0750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:08.915274 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 35/64] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:08.939690 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f33f41-9df9-40ea-8920-f6705c4a2c57 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:08.984556 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:09.014045 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 30/65] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:55:09.055731 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a34315-ad94-4152-803f-0379dbed0750 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:09.091961 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:09.126262 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f33f41-9df9-40ea-8920-f6705c4a2c57 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:09.187625 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd54130-298b-4090-b767-29a31191f839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:09.188421 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd54130-298b-4090-b767-29a31191f839 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:09.188744 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd54130-298b-4090-b767-29a31191f839 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:09.189030 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd54130-298b-4090-b767-29a31191f839 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:09.204782 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd54130-298b-4090-b767-29a31191f839 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:09.213584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-8fd54130-298b-4090-b767-29a31191f839 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:55:09.213584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:09.213584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:09.213584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:09.213584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:09.213584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:09.213584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:09.213584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:09.213584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:09.213584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:09.213584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:09.213584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:09.213584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:09.213584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:09.213584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:09.213584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:09.214648 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:09.214648 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:09.214648 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:09.214648 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:09.214648 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:09.214648 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:09.214648 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:09.214648 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:09.214648 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:09.214648 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:09.214648 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:09.214648 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:09.214648 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:09.214648 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:09.214648 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:09.214648 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:09.215670 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:09.215670 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:09.215670 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:09.215670 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:09.215670 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:09.215670 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:09.215670 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:09.215670 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:09.215670 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:09.215670 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:09.215670 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:55:09.215670 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:55:09.216401 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a34315-ad94-4152-803f-0379dbed0750 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:09.218958 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 36/66] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 13:55:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:09.220339 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cdf048-56b7-4ea8-8980-4c27353824bf None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:09.220808 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cdf048-56b7-4ea8-8980-4c27353824bf None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:09.221232 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cdf048-56b7-4ea8-8980-4c27353824bf None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:09.226934 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 37/67] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:09.229078 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296db89-cced-4b6a-9e1e-e09a3510d9a4 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:09.229513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296db89-cced-4b6a-9e1e-e09a3510d9a4 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:09.229965 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296db89-cced-4b6a-9e1e-e09a3510d9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:09.253039 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-7296db89-cced-4b6a-9e1e-e09a3510d9a4 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:09.255000 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-7296db89-cced-4b6a-9e1e-e09a3510d9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:09.261517 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c49791-f7f3-47aa-b1fe-ac4a000ef184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:09.262359 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c49791-f7f3-47aa-b1fe-ac4a000ef184 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:09.262825 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c49791-f7f3-47aa-b1fe-ac4a000ef184 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:09.263286 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c49791-f7f3-47aa-b1fe-ac4a000ef184 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:09.275437 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c49791-f7f3-47aa-b1fe-ac4a000ef184 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:09.281199 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-29c49791-f7f3-47aa-b1fe-ac4a000ef184 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:55:09.281199 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:09.281199 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:09.281199 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:09.281199 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:09.281199 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:09.281199 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:09.281199 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:09.281199 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:09.281199 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:09.281199 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:09.281199 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:09.281199 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:09.281199 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:09.281199 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:09.281199 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:09.282247 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:09.282247 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:09.282247 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:09.282247 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:09.282247 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:09.282247 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:09.282247 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:09.282247 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:09.282247 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:09.282247 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:09.282247 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:09.282247 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:09.282247 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:09.282247 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:09.282247 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:09.282247 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:09.283173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:09.283173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:09.283173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:09.283173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:09.283173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:09.283173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:09.283173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:09.283173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:09.283173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:09.283173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:09.283173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:55:09.283173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:55:09.285533 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 31/68] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:09.286498 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c49791-f7f3-47aa-b1fe-ac4a000ef184 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:09.348714 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 38/69] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:09.349526 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7296db89-cced-4b6a-9e1e-e09a3510d9a4 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:09.365143 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:09.379344 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:09.434507 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c49791-f7f3-47aa-b1fe-ac4a000ef184 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:09.489529 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe4e77-c895-4cb4-9ae4-cf2f87d3edd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:09.490676 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe4e77-c895-4cb4-9ae4-cf2f87d3edd5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:09.490972 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe4e77-c895-4cb4-9ae4-cf2f87d3edd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:09.491385 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe4e77-c895-4cb4-9ae4-cf2f87d3edd5 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:09.492137 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbe4e77-c895-4cb4-9ae4-cf2f87d3edd5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:09.492743 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbe4e77-c895-4cb4-9ae4-cf2f87d3edd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:09.494902 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbe4e77-c895-4cb4-9ae4-cf2f87d3edd5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:09.496591 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7296db89-cced-4b6a-9e1e-e09a3510d9a4 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:09.511035 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 32/70] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:09.511035 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc06418-0440-441f-9b63-ec41e21cf755 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:09.511035 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc06418-0440-441f-9b63-ec41e21cf755 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:09.511035 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc06418-0440-441f-9b63-ec41e21cf755 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:09.511035 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 33/71] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:09.519955 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc06418-0440-441f-9b63-ec41e21cf755 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:09.549528 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f5ca6-f633-4335-8d11-7327bc3721db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:09.550169 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f5ca6-f633-4335-8d11-7327bc3721db None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:09.550445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f5ca6-f633-4335-8d11-7327bc3721db None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:09.550721 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f5ca6-f633-4335-8d11-7327bc3721db None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:09.551465 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f5ca6-f633-4335-8d11-7327bc3721db None admin] RBAC: Adding query filter params (name=member) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:09.551770 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f5ca6-f633-4335-8d11-7327bc3721db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:09.553642 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f5ca6-f633-4335-8d11-7327bc3721db None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:09.563137 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 39/72] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 13:55:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:09.563984 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f5ca6-f633-4335-8d11-7327bc3721db None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:09.651697 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc06418-0440-441f-9b63-ec41e21cf755 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:09.652260 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:09.680823 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f5ca6-f633-4335-8d11-7327bc3721db None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:09.691369 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548aa67d-7511-47b7-8b65-3dddac137cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:09.692026 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548aa67d-7511-47b7-8b65-3dddac137cff None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:09.692345 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548aa67d-7511-47b7-8b65-3dddac137cff None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:09.692643 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548aa67d-7511-47b7-8b65-3dddac137cff None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:09.695349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:09.703866 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548aa67d-7511-47b7-8b65-3dddac137cff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:09.709239 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-548aa67d-7511-47b7-8b65-3dddac137cff None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:55:09.709239 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:09.709239 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:09.709239 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:09.709239 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:09.709239 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:09.709239 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:09.709239 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:09.709239 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:09.709239 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:09.709239 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:09.709239 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:09.709239 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:09.709239 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:09.709239 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:09.709239 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:09.710050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:09.710050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:09.710050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:09.710050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:09.710050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:09.710050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:09.710050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:09.710050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:09.710050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:09.710050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:09.710050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:09.710050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:09.710050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:09.710050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:09.710050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:09.710050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:09.710774 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:09.710774 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:09.710774 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:09.710774 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:09.710774 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:09.710774 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:09.710774 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:09.710774 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:09.710774 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:09.710774 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:09.710774 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:55:09.710774 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:55:09.710774 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 34/73] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 13:55:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:09.711929 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15168fe3-9046-41b3-8e52-aa57636c4a57 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:09.711929 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15168fe3-9046-41b3-8e52-aa57636c4a57 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:09.712207 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15168fe3-9046-41b3-8e52-aa57636c4a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:09.722159 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d3fb5-7598-43b7-a81d-b5c00e251bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:09.722815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d3fb5-7598-43b7-a81d-b5c00e251bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:09.723102 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d3fb5-7598-43b7-a81d-b5c00e251bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:09.723385 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d3fb5-7598-43b7-a81d-b5c00e251bb3 None admin] PATH_INFO: `/v3/users/7d5a51326470403490a7155c74035d21` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:09.724610 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589d3fb5-7598-43b7-a81d-b5c00e251bb3 None admin] RBAC: Authorizing `identity:get_user(user_id=7d5a51326470403490a7155c74035d21)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:09.726097 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589d3fb5-7598-43b7-a81d-b5c00e251bb3 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:09.727692 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 40/74] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:09 2022] GET /identity/v3/users/7d5a51326470403490a7155c74035d21 => generated 247 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:09.728843 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb69e17-8055-49f6-b237-8fab64fd4365 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:09.729048 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-15168fe3-9046-41b3-8e52-aa57636c4a57 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:09.729139 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb69e17-8055-49f6-b237-8fab64fd4365 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:09.729392 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb69e17-8055-49f6-b237-8fab64fd4365 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:09.730241 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-15168fe3-9046-41b3-8e52-aa57636c4a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:09.730637 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 41/75] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:09.731375 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb69e17-8055-49f6-b237-8fab64fd4365 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:09.801119 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 35/76] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:09.802083 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15168fe3-9046-41b3-8e52-aa57636c4a57 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:09.852882 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb69e17-8055-49f6-b237-8fab64fd4365 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:09.887240 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99986217-edc3-4129-b799-4356294ae516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:09.887923 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99986217-edc3-4129-b799-4356294ae516 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:09.888233 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99986217-edc3-4129-b799-4356294ae516 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:09.888543 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99986217-edc3-4129-b799-4356294ae516 None admin] PATH_INFO: `/v3/users/b3e958a355f0429dab88a0bfd6dd81e6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:09.905176 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99986217-edc3-4129-b799-4356294ae516 None admin] RBAC: Authorizing `identity:get_user(user_id=b3e958a355f0429dab88a0bfd6dd81e6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:09.906567 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99986217-edc3-4129-b799-4356294ae516 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:09.908251 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 42/77] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:09 2022] GET /identity/v3/users/b3e958a355f0429dab88a0bfd6dd81e6 => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:09.909060 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99986217-edc3-4129-b799-4356294ae516 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:09.912366 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:09.921359 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15168fe3-9046-41b3-8e52-aa57636c4a57 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:09.950292 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:09.959816 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33a484b-2d6c-4d35-8a41-af6179d15ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:09.960447 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a484b-2d6c-4d35-8a41-af6179d15ec2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:09.960761 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a484b-2d6c-4d35-8a41-af6179d15ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:09.961037 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a484b-2d6c-4d35-8a41-af6179d15ec2 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:09.961801 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33a484b-2d6c-4d35-8a41-af6179d15ec2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:09.962107 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33a484b-2d6c-4d35-8a41-af6179d15ec2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:09.963489 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33a484b-2d6c-4d35-8a41-af6179d15ec2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:09.971524 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 36/78] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:55:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:09.972797 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee584985-f75c-4c39-97a9-5a57ca58c348 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:09.973094 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee584985-f75c-4c39-97a9-5a57ca58c348 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:09.973379 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee584985-f75c-4c39-97a9-5a57ca58c348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:09.990862 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ee584985-f75c-4c39-97a9-5a57ca58c348 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:09.992036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ee584985-f75c-4c39-97a9-5a57ca58c348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:10.020155 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99986217-edc3-4129-b799-4356294ae516 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:10.055665 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0961b1ed-50d1-45cf-99c0-e1099d2136b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:10.056299 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0961b1ed-50d1-45cf-99c0-e1099d2136b3 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:10.056622 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0961b1ed-50d1-45cf-99c0-e1099d2136b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:10.056912 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0961b1ed-50d1-45cf-99c0-e1099d2136b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:10.064121 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 37/79] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:10.065010 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee584985-f75c-4c39-97a9-5a57ca58c348 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:10.135893 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0961b1ed-50d1-45cf-99c0-e1099d2136b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:10.137378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0961b1ed-50d1-45cf-99c0-e1099d2136b3 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:10.139244 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:10.163877 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee584985-f75c-4c39-97a9-5a57ca58c348 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:10.182159 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:10.198793 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4eb580-5e34-403d-917a-82b7c549c76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:10.199431 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4eb580-5e34-403d-917a-82b7c549c76e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:10.199722 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4eb580-5e34-403d-917a-82b7c549c76e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:10.200007 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4eb580-5e34-403d-917a-82b7c549c76e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:10.210110 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4eb580-5e34-403d-917a-82b7c549c76e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:10.214964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-cd4eb580-5e34-403d-917a-82b7c549c76e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:55:10.214964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:10.214964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:10.214964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:10.214964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:10.214964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:10.214964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:10.214964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:10.214964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:10.214964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:10.214964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:10.214964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:10.214964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:10.214964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:10.214964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:10.214964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:10.215778 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:10.215778 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:10.215778 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:10.215778 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:10.215778 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:10.215778 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:10.215778 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:10.215778 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:10.215778 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:10.215778 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:10.215778 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:10.215778 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:10.215778 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:10.215778 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:10.215778 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:10.215778 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:10.217482 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:10.217482 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:10.217482 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:10.217482 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:10.217482 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:10.217482 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:10.217482 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:10.217482 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:10.217482 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:10.217482 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:10.217482 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:55:10.217482 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:55:10.217482 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 38/80] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:10.217482 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4eb580-5e34-403d-917a-82b7c549c76e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:10.239634 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 43/81] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:55:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:55:10.240842 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0961b1ed-50d1-45cf-99c0-e1099d2136b3 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:10.321729 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4eb580-5e34-403d-917a-82b7c549c76e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:10.340925 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0961b1ed-50d1-45cf-99c0-e1099d2136b3 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:10.342944 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:10.354715 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbbcb13-441a-4e26-835f-d7a1558f6ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:10.355347 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbcb13-441a-4e26-835f-d7a1558f6ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:10.355613 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbcb13-441a-4e26-835f-d7a1558f6ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:10.355894 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbcb13-441a-4e26-835f-d7a1558f6ab0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:10.376747 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8933d7-e510-4202-9274-a6088643dc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:10.377395 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8933d7-e510-4202-9274-a6088643dc29 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:10.377684 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8933d7-e510-4202-9274-a6088643dc29 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:10.377970 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8933d7-e510-4202-9274-a6088643dc29 None admin] PATH_INFO: `/v3/users/b3e958a355f0429dab88a0bfd6dd81e6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:10.379155 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8933d7-e510-4202-9274-a6088643dc29 None admin] RBAC: Authorizing `identity:get_user(user_id=b3e958a355f0429dab88a0bfd6dd81e6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:10.380658 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8933d7-e510-4202-9274-a6088643dc29 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:10.382262 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 44/82] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:10 2022] GET /identity/v3/users/b3e958a355f0429dab88a0bfd6dd81e6 => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:10.383030 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8933d7-e510-4202-9274-a6088643dc29 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:10.403165 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:10.425827 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbbcb13-441a-4e26-835f-d7a1558f6ab0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:10.427250 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbbcb13-441a-4e26-835f-d7a1558f6ab0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:10.480640 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8933d7-e510-4202-9274-a6088643dc29 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:10.513894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b0407-dc27-4fb4-bf15-b02463c6b909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:10.514538 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b0407-dc27-4fb4-bf15-b02463c6b909 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:10.514824 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b0407-dc27-4fb4-bf15-b02463c6b909 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:10.515104 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b0407-dc27-4fb4-bf15-b02463c6b909 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:10.516253 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16b0407-dc27-4fb4-bf15-b02463c6b909 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:10.517817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16b0407-dc27-4fb4-bf15-b02463c6b909 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:10.519589 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 45/83] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:55:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:10.520229 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b0407-dc27-4fb4-bf15-b02463c6b909 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:10.537997 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 39/84] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:55:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:55:10.537997 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbbcb13-441a-4e26-835f-d7a1558f6ab0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:10.563437 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:10.610603 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:10.623253 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b0407-dc27-4fb4-bf15-b02463c6b909 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:10.632239 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbbcb13-441a-4e26-835f-d7a1558f6ab0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:10.659398 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b0aed-51a9-4530-80e6-f51a753457ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:10.660020 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b0aed-51a9-4530-80e6-f51a753457ea None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:10.660311 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b0aed-51a9-4530-80e6-f51a753457ea None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:10.660615 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b0aed-51a9-4530-80e6-f51a753457ea None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:10.661343 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2b0aed-51a9-4530-80e6-f51a753457ea None admin] RBAC: Adding query filter params (name=member) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:10.661647 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2b0aed-51a9-4530-80e6-f51a753457ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:10.663036 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2b0aed-51a9-4530-80e6-f51a753457ea None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:10.669106 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239ac9e-273c-4250-b21d-efa97423726b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:10.670250 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239ac9e-273c-4250-b21d-efa97423726b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:10.670536 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239ac9e-273c-4250-b21d-efa97423726b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:10.670814 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239ac9e-273c-4250-b21d-efa97423726b None admin] PATH_INFO: `/v3/projects/516ee04b71684e5885fa9a08b821fb87` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:10.671790 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 46/85] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 13:55:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:10.671995 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8239ac9e-273c-4250-b21d-efa97423726b None admin] RBAC: Authorizing `identity:get_project(project_id=516ee04b71684e5885fa9a08b821fb87)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:10.672606 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b0aed-51a9-4530-80e6-f51a753457ea None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:10.673473 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8239ac9e-273c-4250-b21d-efa97423726b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:10.675126 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 40/86] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:10 2022] GET /identity/v3/projects/516ee04b71684e5885fa9a08b821fb87 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:10.676131 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239ac9e-273c-4250-b21d-efa97423726b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:10.767979 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:10.771398 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b0aed-51a9-4530-80e6-f51a753457ea None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:10.777203 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239ac9e-273c-4250-b21d-efa97423726b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:10.806614 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d1bd6-e6c3-44ef-9a58-898da55f3b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:10.807325 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d1bd6-e6c3-44ef-9a58-898da55f3b5b None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:10.807627 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d1bd6-e6c3-44ef-9a58-898da55f3b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:10.807915 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d1bd6-e6c3-44ef-9a58-898da55f3b5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:10.811194 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3e488e-70dd-430f-961f-50e1a0564f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:10.811828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e488e-70dd-430f-961f-50e1a0564f06 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:10.812116 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e488e-70dd-430f-961f-50e1a0564f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:10.812400 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e488e-70dd-430f-961f-50e1a0564f06 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:10.813121 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3e488e-70dd-430f-961f-50e1a0564f06 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:10.814573 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3e488e-70dd-430f-961f-50e1a0564f06 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:10.841763 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 41/87] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 13:55:10 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:55:10.843046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3e488e-70dd-430f-961f-50e1a0564f06 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:10.845963 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:10.887927 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1d1bd6-e6c3-44ef-9a58-898da55f3b5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:10.889488 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1d1bd6-e6c3-44ef-9a58-898da55f3b5b None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:10.964367 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3e488e-70dd-430f-961f-50e1a0564f06 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:11.003750 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:11.006602 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf929414-a57e-4928-9a18-3b20d6aeb23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:11.007439 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf929414-a57e-4928-9a18-3b20d6aeb23c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:11.007667 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 47/88] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:55:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:55:11.007766 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf929414-a57e-4928-9a18-3b20d6aeb23c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:11.008749 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf929414-a57e-4928-9a18-3b20d6aeb23c None admin] PATH_INFO: `/v3/projects/516ee04b71684e5885fa9a08b821fb87` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:11.010068 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d1bd6-e6c3-44ef-9a58-898da55f3b5b None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:11.010173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf929414-a57e-4928-9a18-3b20d6aeb23c None admin] RBAC: Authorizing `identity:get_project(project_id=516ee04b71684e5885fa9a08b821fb87)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:11.011164 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf929414-a57e-4928-9a18-3b20d6aeb23c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:11.014244 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 42/89] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:10 2022] GET /identity/v3/projects/516ee04b71684e5885fa9a08b821fb87 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:11.019107 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf929414-a57e-4928-9a18-3b20d6aeb23c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:11.096531 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:11.131164 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d1bd6-e6c3-44ef-9a58-898da55f3b5b None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:11.135316 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf929414-a57e-4928-9a18-3b20d6aeb23c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:11.172210 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21236084-9333-4b1f-8569-dbd28c6f1b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:11.172920 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21236084-9333-4b1f-8569-dbd28c6f1b73 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:11.173181 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21236084-9333-4b1f-8569-dbd28c6f1b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:11.173456 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21236084-9333-4b1f-8569-dbd28c6f1b73 None admin] PATH_INFO: `/v3/users/b3e958a355f0429dab88a0bfd6dd81e6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:11.175482 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21236084-9333-4b1f-8569-dbd28c6f1b73 None admin] RBAC: Authorizing `identity:get_user(user_id=b3e958a355f0429dab88a0bfd6dd81e6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:11.176090 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21236084-9333-4b1f-8569-dbd28c6f1b73 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:11.178235 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76df3e-6bf5-4834-82a9-4ef75acd5dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:11.178361 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 48/90] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:11 2022] GET /identity/v3/users/b3e958a355f0429dab88a0bfd6dd81e6 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:11.178433 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76df3e-6bf5-4834-82a9-4ef75acd5dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:11.178433 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76df3e-6bf5-4834-82a9-4ef75acd5dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:11.178571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21236084-9333-4b1f-8569-dbd28c6f1b73 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:11.178656 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76df3e-6bf5-4834-82a9-4ef75acd5dc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:11.179632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76df3e-6bf5-4834-82a9-4ef75acd5dc2 None admin] RBAC: Adding query filter params (role.id=58f91e3a13664dabb18c6227a1658fa6, scope.project.id=516ee04b71684e5885fa9a08b821fb87, user.id=7d5a51326470403490a7155c74035d21) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:11.180009 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76df3e-6bf5-4834-82a9-4ef75acd5dc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:11.181794 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76df3e-6bf5-4834-82a9-4ef75acd5dc2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:11.193776 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 43/91] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:55:11 2022] GET /identity/v3/role_assignments?user.id=7d5a51326470403490a7155c74035d21&scope.project.id=516ee04b71684e5885fa9a08b821fb87&role.id=58f91e3a13664dabb18c6227a1658fa6 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:11.196774 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76df3e-6bf5-4834-82a9-4ef75acd5dc2 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:11.264050 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:11.306043 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21236084-9333-4b1f-8569-dbd28c6f1b73 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:11.330664 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76df3e-6bf5-4834-82a9-4ef75acd5dc2 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:11.345687 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a3210-c9bc-434f-947f-9f9aaa3e6659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:11.346468 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a3210-c9bc-434f-947f-9f9aaa3e6659 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:11.346799 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a3210-c9bc-434f-947f-9f9aaa3e6659 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:11.347079 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a3210-c9bc-434f-947f-9f9aaa3e6659 None admin] PATH_INFO: `/v3/projects/516ee04b71684e5885fa9a08b821fb87` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:11.348453 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4a3210-c9bc-434f-947f-9f9aaa3e6659 None admin] RBAC: Authorizing `identity:get_project(project_id=516ee04b71684e5885fa9a08b821fb87)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:11.350037 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4a3210-c9bc-434f-947f-9f9aaa3e6659 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:11.351884 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 49/92] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:11 2022] GET /identity/v3/projects/516ee04b71684e5885fa9a08b821fb87 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:11.355440 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a3210-c9bc-434f-947f-9f9aaa3e6659 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:11.375946 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824215a-d7f6-4d03-b5fc-b912c14a898e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:11.377139 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824215a-d7f6-4d03-b5fc-b912c14a898e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:11.377678 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824215a-d7f6-4d03-b5fc-b912c14a898e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:11.378157 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824215a-d7f6-4d03-b5fc-b912c14a898e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:11.379165 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0824215a-d7f6-4d03-b5fc-b912c14a898e None admin] RBAC: Adding query filter params (role.id=926b9f36c24847c582c84af904250abc, scope.project.id=516ee04b71684e5885fa9a08b821fb87, user.id=b3e958a355f0429dab88a0bfd6dd81e6) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:11.379730 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0824215a-d7f6-4d03-b5fc-b912c14a898e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:11.382900 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0824215a-d7f6-4d03-b5fc-b912c14a898e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:11.385824 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:11.391225 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 44/93] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:55:11 2022] GET /identity/v3/role_assignments?user.id=b3e958a355f0429dab88a0bfd6dd81e6&scope.project.id=516ee04b71684e5885fa9a08b821fb87&role.id=926b9f36c24847c582c84af904250abc => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:11.392611 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824215a-d7f6-4d03-b5fc-b912c14a898e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:11.479828 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a3210-c9bc-434f-947f-9f9aaa3e6659 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:11.520152 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc35f16a-ed49-4d01-b95a-525b4f6c909b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:11.520769 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35f16a-ed49-4d01-b95a-525b4f6c909b None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:11.521040 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35f16a-ed49-4d01-b95a-525b4f6c909b None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:11.521315 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35f16a-ed49-4d01-b95a-525b4f6c909b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:11.522520 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824215a-d7f6-4d03-b5fc-b912c14a898e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:11.525506 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:11.562512 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885b1d61-534f-4407-b206-84d9c623d764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:11.563598 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885b1d61-534f-4407-b206-84d9c623d764 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:11.564082 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885b1d61-534f-4407-b206-84d9c623d764 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:11.564651 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885b1d61-534f-4407-b206-84d9c623d764 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:11.565687 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885b1d61-534f-4407-b206-84d9c623d764 None admin] RBAC: Adding query filter params (role.id=47e170213d8049a28f8b8a0d736fdf6b, scope.project.id=516ee04b71684e5885fa9a08b821fb87, user.id=b3e958a355f0429dab88a0bfd6dd81e6) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:11.566221 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885b1d61-534f-4407-b206-84d9c623d764 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:11.567930 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885b1d61-534f-4407-b206-84d9c623d764 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:11.578886 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 45/94] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:55:11 2022] GET /identity/v3/role_assignments?user.id=b3e958a355f0429dab88a0bfd6dd81e6&scope.project.id=516ee04b71684e5885fa9a08b821fb87&role.id=47e170213d8049a28f8b8a0d736fdf6b => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:11.615662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc35f16a-ed49-4d01-b95a-525b4f6c909b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:11.617201 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc35f16a-ed49-4d01-b95a-525b4f6c909b None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:11.729726 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 50/95] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:55:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:55:11.735796 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885b1d61-534f-4407-b206-84d9c623d764 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:11.817213 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:11.865256 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885b1d61-534f-4407-b206-84d9c623d764 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:11.902395 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb7383-acd7-4cd7-a07c-a3b71a0aa607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:11.903050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb7383-acd7-4cd7-a07c-a3b71a0aa607 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:11.903337 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb7383-acd7-4cd7-a07c-a3b71a0aa607 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:11.903617 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb7383-acd7-4cd7-a07c-a3b71a0aa607 None admin] PATH_INFO: `/v3/projects/ab213fe217b44205a8e298d3ec240af6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:11.905450 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cb7383-acd7-4cd7-a07c-a3b71a0aa607 None admin] RBAC: Authorizing `identity:get_project(project_id=ab213fe217b44205a8e298d3ec240af6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:11.907093 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cb7383-acd7-4cd7-a07c-a3b71a0aa607 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:11.909113 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 46/96] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:11 2022] GET /identity/v3/projects/ab213fe217b44205a8e298d3ec240af6 => generated 312 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:11.916016 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc35f16a-ed49-4d01-b95a-525b4f6c909b None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:11.953319 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:12.024740 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc35f16a-ed49-4d01-b95a-525b4f6c909b None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:12.061370 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad9e65f-f79e-43ef-9547-013f3a3d335c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:12.062049 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9e65f-f79e-43ef-9547-013f3a3d335c None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:12.062337 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9e65f-f79e-43ef-9547-013f3a3d335c None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:12.062614 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9e65f-f79e-43ef-9547-013f3a3d335c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:12.063421 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad9e65f-f79e-43ef-9547-013f3a3d335c None admin] RBAC: Adding query filter params (role.id=926b9f36c24847c582c84af904250abc, scope.project.id=ab213fe217b44205a8e298d3ec240af6, user.id=b3e958a355f0429dab88a0bfd6dd81e6) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:12.063730 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad9e65f-f79e-43ef-9547-013f3a3d335c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:12.065217 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad9e65f-f79e-43ef-9547-013f3a3d335c None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:12.072506 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 51/97] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:55:11 2022] GET /identity/v3/role_assignments?user.id=b3e958a355f0429dab88a0bfd6dd81e6&scope.project.id=ab213fe217b44205a8e298d3ec240af6&role.id=926b9f36c24847c582c84af904250abc => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:14.053380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f1abd-8f66-4eb8-aca7-d31d913e748d None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:14.053873 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f1abd-8f66-4eb8-aca7-d31d913e748d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:14.053945 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f1abd-8f66-4eb8-aca7-d31d913e748d None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:14.055468 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 47/98] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:14.060536 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af3195-fcb4-4159-bca0-d9e9ec7e1a79 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:14.061307 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af3195-fcb4-4159-bca0-d9e9ec7e1a79 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:14.061803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af3195-fcb4-4159-bca0-d9e9ec7e1a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:14.085536 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-18af3195-fcb4-4159-bca0-d9e9ec7e1a79 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:14.087026 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-18af3195-fcb4-4159-bca0-d9e9ec7e1a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:14.098641 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd26838-1b05-46fa-879a-07c0c0f193fe None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:14.098938 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd26838-1b05-46fa-879a-07c0c0f193fe None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:14.099227 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd26838-1b05-46fa-879a-07c0c0f193fe None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:14.100612 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 48/99] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:14.105402 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a3297-0014-4929-92a1-db9f5179ba23 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:14.105695 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a3297-0014-4929-92a1-db9f5179ba23 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:14.105982 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a3297-0014-4929-92a1-db9f5179ba23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:14.154126 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-616a3297-0014-4929-92a1-db9f5179ba23 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:14.155376 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-616a3297-0014-4929-92a1-db9f5179ba23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:14.165834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 52/100] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:14.201568 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af3195-fcb4-4159-bca0-d9e9ec7e1a79 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:14.259681 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 49/101] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 01 13:55:14.267962 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:14.299418 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a3297-0014-4929-92a1-db9f5179ba23 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:14.402158 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af3195-fcb4-4159-bca0-d9e9ec7e1a79 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:14.409521 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:14.450256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87355bd-8436-4e3a-ac40-e0b554dabc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:14.451918 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87355bd-8436-4e3a-ac40-e0b554dabc0f None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:14.452215 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87355bd-8436-4e3a-ac40-e0b554dabc0f None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:14.452803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87355bd-8436-4e3a-ac40-e0b554dabc0f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:14.454249 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87355bd-8436-4e3a-ac40-e0b554dabc0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:14.456395 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87355bd-8436-4e3a-ac40-e0b554dabc0f None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:14.457441 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a3297-0014-4929-92a1-db9f5179ba23 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:14.458046 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 53/102] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:55:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:14.461095 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f96e6-ef0e-4cbb-863b-c6fea352e36e None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:14.461382 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f96e6-ef0e-4cbb-863b-c6fea352e36e None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:14.461664 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f96e6-ef0e-4cbb-863b-c6fea352e36e None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:14.463205 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 54/103] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:14.464323 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85d475-200a-49e5-8eae-85d522920d77 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:14.464596 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85d475-200a-49e5-8eae-85d522920d77 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:14.464878 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85d475-200a-49e5-8eae-85d522920d77 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:14.466641 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 55/104] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:14.467335 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec85d475-200a-49e5-8eae-85d522920d77 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:14.504613 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc82e46-6bdf-41b7-8977-194398a67f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:14.505269 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc82e46-6bdf-41b7-8977-194398a67f7d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:14.505558 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc82e46-6bdf-41b7-8977-194398a67f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:14.505847 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc82e46-6bdf-41b7-8977-194398a67f7d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:14.517698 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc82e46-6bdf-41b7-8977-194398a67f7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:14.527837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-2cc82e46-6bdf-41b7-8977-194398a67f7d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:55:14.527837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:14.527837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:14.527837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:14.527837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:14.527837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:14.527837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:14.527837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:14.527837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:14.527837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:14.527837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:14.527837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:14.527837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:14.527837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:14.527837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:14.527837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:14.528744 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:14.528744 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:14.528744 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:14.528744 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:14.528744 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:14.528744 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:14.528744 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:14.528744 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:14.528744 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:14.528744 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:14.528744 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:14.528744 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:14.528744 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:14.528744 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:14.528744 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:14.528744 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:14.529538 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:14.529538 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:14.529538 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:14.529538 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:14.529538 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:14.529538 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:14.529538 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:14.529538 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:14.529538 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:14.529538 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:14.529538 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:55:14.529538 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:55:14.532617 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 50/105] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 13:55:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:14.535478 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d73bb7-458a-4cf3-a857-3b4531701abc None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:14.535978 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d73bb7-458a-4cf3-a857-3b4531701abc None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:14.536869 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d73bb7-458a-4cf3-a857-3b4531701abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:14.553974 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d7d73bb7-458a-4cf3-a857-3b4531701abc None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:14.555878 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d7d73bb7-458a-4cf3-a857-3b4531701abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:14.578709 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:14.585177 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec85d475-200a-49e5-8eae-85d522920d77 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:14.621767 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb03726-15cf-4acd-8150-fa4097cf5a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:14.622904 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb03726-15cf-4acd-8150-fa4097cf5a6b None admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:14.623197 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb03726-15cf-4acd-8150-fa4097cf5a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:14.623652 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb03726-15cf-4acd-8150-fa4097cf5a6b None admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:14.624215 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb03726-15cf-4acd-8150-fa4097cf5a6b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:14.625658 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb03726-15cf-4acd-8150-fa4097cf5a6b None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:14.636747 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 51/106] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:14.636747 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6dbad-6f8c-4eb6-bb0a-160a8270ce0c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:14.636747 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6dbad-6f8c-4eb6-bb0a-160a8270ce0c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:14.636747 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6dbad-6f8c-4eb6-bb0a-160a8270ce0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:14.652706 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a9c6dbad-6f8c-4eb6-bb0a-160a8270ce0c None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:14.654075 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a9c6dbad-6f8c-4eb6-bb0a-160a8270ce0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:14.672511 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 56/107] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 13:55:14 2022] POST /identity/v3/projects => generated 302 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:55:14.674120 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b30b1-c564-4ed1-bc25-1ff59c89489b None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:14.674419 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b30b1-c564-4ed1-bc25-1ff59c89489b None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:14.674708 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b30b1-c564-4ed1-bc25-1ff59c89489b None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:14.676336 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 57/108] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:14.681426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b30b1-c564-4ed1-bc25-1ff59c89489b None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:14.729084 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:14.769161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 52/109] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:14.772046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c6dbad-6f8c-4eb6-bb0a-160a8270ce0c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:14.845740 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b30b1-c564-4ed1-bc25-1ff59c89489b None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:14.891818 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc10388-8d21-42e4-8f04-8c5269728a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:14.892451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc10388-8d21-42e4-8f04-8c5269728a72 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:14.892802 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc10388-8d21-42e4-8f04-8c5269728a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:14.893085 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc10388-8d21-42e4-8f04-8c5269728a72 None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:14.893832 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc10388-8d21-42e4-8f04-8c5269728a72 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:14.894120 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc10388-8d21-42e4-8f04-8c5269728a72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:14.895505 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc10388-8d21-42e4-8f04-8c5269728a72 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:14.906224 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c6dbad-6f8c-4eb6-bb0a-160a8270ce0c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:14.906373 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 58/110] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:14.907582 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4d81a-b9c9-4923-a0fe-79fd5a127bc5 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:14.907875 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4d81a-b9c9-4923-a0fe-79fd5a127bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:14.908155 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4d81a-b9c9-4923-a0fe-79fd5a127bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:14.927133 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-e1f4d81a-b9c9-4923-a0fe-79fd5a127bc5 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:14.928391 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-e1f4d81a-b9c9-4923-a0fe-79fd5a127bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:14.936942 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:14.948135 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ccfc12-6261-4229-99bb-95f46a32752d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:14.949086 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ccfc12-6261-4229-99bb-95f46a32752d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:14.949630 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ccfc12-6261-4229-99bb-95f46a32752d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:14.950071 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ccfc12-6261-4229-99bb-95f46a32752d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:14.962047 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ccfc12-6261-4229-99bb-95f46a32752d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:14.967866 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-b7ccfc12-6261-4229-99bb-95f46a32752d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 13:55:14.967866 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:14.967866 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:14.967866 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:14.967866 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:14.967866 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:14.967866 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:14.967866 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:14.967866 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:14.967866 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:14.967866 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:14.967866 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:14.967866 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:14.967866 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:14.967866 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:14.967866 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:14.968868 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:14.968868 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:14.968868 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:14.968868 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:14.968868 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:14.968868 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:14.968868 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:14.968868 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:14.968868 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:14.968868 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:14.968868 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:14.968868 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:14.968868 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:14.968868 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:14.968868 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:14.968868 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:14.969614 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:14.969614 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:14.969614 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:14.969614 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:14.969614 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:14.969614 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:14.969614 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:14.969614 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:14.969614 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:14.969614 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:14.969614 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 13:55:14.969614 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:55:14.972240 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 53/111] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:14.973316 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ccfc12-6261-4229-99bb-95f46a32752d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:15.013240 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 59/112] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:15.021217 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4d81a-b9c9-4923-a0fe-79fd5a127bc5 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:15.027929 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:15.110333 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ccfc12-6261-4229-99bb-95f46a32752d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:15.129904 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4d81a-b9c9-4923-a0fe-79fd5a127bc5 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:15.148637 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afbe82-ad9d-4fe1-b463-6f8349af3362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:15.149396 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afbe82-ad9d-4fe1-b463-6f8349af3362 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:15.149681 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afbe82-ad9d-4fe1-b463-6f8349af3362 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:15.149959 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afbe82-ad9d-4fe1-b463-6f8349af3362 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:15.161701 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15afbe82-ad9d-4fe1-b463-6f8349af3362 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:15.166909 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-15afbe82-ad9d-4fe1-b463-6f8349af3362 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:55:15.166909 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:15.166909 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:15.166909 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:15.166909 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:15.166909 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:15.166909 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:15.166909 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:15.166909 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:15.166909 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:15.166909 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:15.166909 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:15.166909 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:15.166909 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:15.166909 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:15.166909 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:15.167747 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:15.167747 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:15.167747 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:15.167747 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:15.167747 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:15.167747 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:15.167747 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:15.167747 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:15.167747 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:15.167747 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:15.167747 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:15.167747 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:15.167747 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:15.167747 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:15.167747 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:15.167747 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:15.168516 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:15.168516 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:15.168516 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:15.168516 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:15.168516 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:15.168516 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:15.168516 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:15.168516 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:15.168516 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:15.168516 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:15.168516 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:55:15.168516 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:55:15.172018 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dea1a8-2c66-4331-b496-f3266fec8578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:15.172795 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 54/113] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 13:55:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:15.173167 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dea1a8-2c66-4331-b496-f3266fec8578 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:15.173646 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dea1a8-2c66-4331-b496-f3266fec8578 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:15.174118 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dea1a8-2c66-4331-b496-f3266fec8578 None admin] PATH_INFO: `/v3/users/7d5a51326470403490a7155c74035d21` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:15.174295 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afbe82-ad9d-4fe1-b463-6f8349af3362 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:15.176024 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dea1a8-2c66-4331-b496-f3266fec8578 None admin] RBAC: Authorizing `identity:get_user(user_id=7d5a51326470403490a7155c74035d21)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:15.177719 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dea1a8-2c66-4331-b496-f3266fec8578 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:15.179670 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 60/114] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:15 2022] GET /identity/v3/users/7d5a51326470403490a7155c74035d21 => generated 247 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:15.180820 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dea1a8-2c66-4331-b496-f3266fec8578 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:15.194668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:15.282495 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:15.286265 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afbe82-ad9d-4fe1-b463-6f8349af3362 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:15.290806 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dea1a8-2c66-4331-b496-f3266fec8578 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:15.322421 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b3df5-35f2-476a-8e03-7dda3aa99ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:15.323047 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b3df5-35f2-476a-8e03-7dda3aa99ebf None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:15.323333 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b3df5-35f2-476a-8e03-7dda3aa99ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:15.323669 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b3df5-35f2-476a-8e03-7dda3aa99ebf None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:15.324331 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710b3df5-35f2-476a-8e03-7dda3aa99ebf None admin] RBAC: Adding query filter params (name=reader) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:15.324664 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710b3df5-35f2-476a-8e03-7dda3aa99ebf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:15.326106 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710b3df5-35f2-476a-8e03-7dda3aa99ebf None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:15.326544 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8654813b-2be6-4e3a-b52a-63e019e567ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:15.327196 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654813b-2be6-4e3a-b52a-63e019e567ba None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:15.327646 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654813b-2be6-4e3a-b52a-63e019e567ba None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:15.327932 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654813b-2be6-4e3a-b52a-63e019e567ba None admin] PATH_INFO: `/v3/roles/member` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:15.336157 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 55/115] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 13:55:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:15.337707 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b3df5-35f2-476a-8e03-7dda3aa99ebf None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:15.343987 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8654813b-2be6-4e3a-b52a-63e019e567ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:15.348935 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-8654813b-2be6-4e3a-b52a-63e019e567ba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:55:15.348935 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:15.348935 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:15.348935 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:15.348935 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:15.348935 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:15.348935 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:15.348935 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:15.348935 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:15.348935 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:15.348935 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:15.348935 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:15.348935 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:15.348935 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:15.348935 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:15.348935 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:15.349840 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:15.349840 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:15.349840 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:15.349840 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:15.349840 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:15.349840 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:15.349840 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:15.349840 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:15.349840 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:15.349840 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:15.349840 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:15.349840 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:15.349840 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:15.349840 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:15.349840 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:15.349840 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:15.350600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:15.350600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:15.350600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:15.350600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:15.350600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:15.350600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:15.350600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:15.350600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:15.350600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:15.350600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:15.350600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:55:15.350600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:55:15.350600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 61/116] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:15.353075 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8654813b-2be6-4e3a-b52a-63e019e567ba None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:15.427942 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:15.460921 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b3df5-35f2-476a-8e03-7dda3aa99ebf None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:15.465098 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8654813b-2be6-4e3a-b52a-63e019e567ba None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:15.497278 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff69636-d195-4e67-b299-18f8266aedf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:15.497852 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff69636-d195-4e67-b299-18f8266aedf6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:15.498144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff69636-d195-4e67-b299-18f8266aedf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:15.498428 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff69636-d195-4e67-b299-18f8266aedf6 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:15.499170 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff69636-d195-4e67-b299-18f8266aedf6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:15.499483 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff69636-d195-4e67-b299-18f8266aedf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:15.500891 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff69636-d195-4e67-b299-18f8266aedf6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:15.504284 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac111c-9e24-40c2-84c6-dc383a8d9671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:15.505166 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac111c-9e24-40c2-84c6-dc383a8d9671 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:15.505628 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac111c-9e24-40c2-84c6-dc383a8d9671 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:15.506108 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac111c-9e24-40c2-84c6-dc383a8d9671 None admin] PATH_INFO: `/v3/projects/516ee04b71684e5885fa9a08b821fb87` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:15.507285 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac111c-9e24-40c2-84c6-dc383a8d9671 None admin] RBAC: Authorizing `identity:get_project(project_id=516ee04b71684e5885fa9a08b821fb87)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:15.508756 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac111c-9e24-40c2-84c6-dc383a8d9671 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:15.514227 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 56/117] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:55:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:15.514357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 62/118] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:15 2022] GET /identity/v3/projects/516ee04b71684e5885fa9a08b821fb87 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:15.514357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac111c-9e24-40c2-84c6-dc383a8d9671 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:15.515753 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b3797-28b6-49cf-badf-d218db57d989 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:15.516067 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b3797-28b6-49cf-badf-d218db57d989 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:15.516353 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b3797-28b6-49cf-badf-d218db57d989 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:15.518331 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 57/119] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:15.518438 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b3797-28b6-49cf-badf-d218db57d989 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:15.546257 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:15.626932 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b3797-28b6-49cf-badf-d218db57d989 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:15.629782 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac111c-9e24-40c2-84c6-dc383a8d9671 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:15.663985 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffe1aa0-c735-4488-8845-a5c52c52e747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:15.664635 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe1aa0-c735-4488-8845-a5c52c52e747 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:15.664929 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe1aa0-c735-4488-8845-a5c52c52e747 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:15.665210 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe1aa0-c735-4488-8845-a5c52c52e747 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:15.665939 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffe1aa0-c735-4488-8845-a5c52c52e747 None admin] RBAC: Adding query filter params (name=member) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:15.666253 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffe1aa0-c735-4488-8845-a5c52c52e747 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:15.667653 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffe1aa0-c735-4488-8845-a5c52c52e747 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:15.668396 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b652239-664b-440c-bb40-e9ee9a8e02fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:15.669254 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b652239-664b-440c-bb40-e9ee9a8e02fc None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:15.669672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b652239-664b-440c-bb40-e9ee9a8e02fc None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:15.670104 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b652239-664b-440c-bb40-e9ee9a8e02fc None admin] PATH_INFO: `/v3/users/305f1669ace14628b1c277f969d44fa8` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:15.677086 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 58/120] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 13:55:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:15.677880 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffe1aa0-c735-4488-8845-a5c52c52e747 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:15.684072 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:15.690010 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b652239-664b-440c-bb40-e9ee9a8e02fc None admin] RBAC: Authorizing `identity:get_user(user_id=305f1669ace14628b1c277f969d44fa8)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:15.691543 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b652239-664b-440c-bb40-e9ee9a8e02fc None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:15.693319 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 63/121] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:15 2022] GET /identity/v3/users/305f1669ace14628b1c277f969d44fa8 => generated 289 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:15.694058 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b652239-664b-440c-bb40-e9ee9a8e02fc None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:15.775204 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:15.791060 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffe1aa0-c735-4488-8845-a5c52c52e747 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:15.818379 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b652239-664b-440c-bb40-e9ee9a8e02fc None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:15.828198 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c0276-5b8c-43ff-84c4-aecd329d41be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:15.828940 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c0276-5b8c-43ff-84c4-aecd329d41be None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:15.829236 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c0276-5b8c-43ff-84c4-aecd329d41be None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:15.829830 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c0276-5b8c-43ff-84c4-aecd329d41be None admin] PATH_INFO: `/v3/users/b3e958a355f0429dab88a0bfd6dd81e6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:15.830733 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c0276-5b8c-43ff-84c4-aecd329d41be None admin] RBAC: Authorizing `identity:get_user(user_id=b3e958a355f0429dab88a0bfd6dd81e6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:15.833019 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c0276-5b8c-43ff-84c4-aecd329d41be None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:15.836540 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 59/122] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:15 2022] GET /identity/v3/users/b3e958a355f0429dab88a0bfd6dd81e6 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:15.838537 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7354d5-df4a-4a8e-951d-45caded9ca79 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:15.839640 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7354d5-df4a-4a8e-951d-45caded9ca79 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:15.840332 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7354d5-df4a-4a8e-951d-45caded9ca79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:15.858785 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d07eb8-dee1-470e-beb4-b6ae6558c98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:15.859586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d07eb8-dee1-470e-beb4-b6ae6558c98c None admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:15.859775 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d07eb8-dee1-470e-beb4-b6ae6558c98c None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:15.860051 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d07eb8-dee1-470e-beb4-b6ae6558c98c None admin] PATH_INFO: `/v3/projects/516ee04b71684e5885fa9a08b821fb87/users/7d5a51326470403490a7155c74035d21/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:15.862192 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d07eb8-dee1-470e-beb4-b6ae6558c98c None admin] RBAC: Authorizing `identity:create_grant(project_id=516ee04b71684e5885fa9a08b821fb87, user_id=7d5a51326470403490a7155c74035d21, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:15.863673 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d07eb8-dee1-470e-beb4-b6ae6558c98c None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:15.864823 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5f7354d5-df4a-4a8e-951d-45caded9ca79 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:15.866651 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5f7354d5-df4a-4a8e-951d-45caded9ca79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:15.877597 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 64/123] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 13:55:15 2022] PUT /identity/v3/projects/516ee04b71684e5885fa9a08b821fb87/users/7d5a51326470403490a7155c74035d21/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:55:15.878446 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d07eb8-dee1-470e-beb4-b6ae6558c98c None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:15.952416 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 60/124] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:15.961613 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7354d5-df4a-4a8e-951d-45caded9ca79 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:15.968912 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:16.028996 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d07eb8-dee1-470e-beb4-b6ae6558c98c None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:16.086840 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:16.087630 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e418e9-1223-4d38-ad51-f26a32ad73ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:16.088300 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e418e9-1223-4d38-ad51-f26a32ad73ef None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:16.088619 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e418e9-1223-4d38-ad51-f26a32ad73ef None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:16.088906 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e418e9-1223-4d38-ad51-f26a32ad73ef None admin] PATH_INFO: `/v3/users/b3e958a355f0429dab88a0bfd6dd81e6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:16.090120 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e418e9-1223-4d38-ad51-f26a32ad73ef None admin] RBAC: Authorizing `identity:get_user(user_id=b3e958a355f0429dab88a0bfd6dd81e6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:16.091662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e418e9-1223-4d38-ad51-f26a32ad73ef None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:16.094904 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 65/125] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:15 2022] GET /identity/v3/users/b3e958a355f0429dab88a0bfd6dd81e6 => generated 275 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:16.095810 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e418e9-1223-4d38-ad51-f26a32ad73ef None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:16.105893 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7354d5-df4a-4a8e-951d-45caded9ca79 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:16.155115 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ef5ef5-5818-45a5-949b-c6fce9282f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:16.155873 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef5ef5-5818-45a5-949b-c6fce9282f72 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:16.156104 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef5ef5-5818-45a5-949b-c6fce9282f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:16.156328 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef5ef5-5818-45a5-949b-c6fce9282f72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:16.224616 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e418e9-1223-4d38-ad51-f26a32ad73ef None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:16.250946 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:16.251893 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ef5ef5-5818-45a5-949b-c6fce9282f72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:16.253708 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ef5ef5-5818-45a5-949b-c6fce9282f72 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:16.263629 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e489cb-0d45-4bd6-8479-a73b0986eebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:16.264416 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e489cb-0d45-4bd6-8479-a73b0986eebf None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:16.264805 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e489cb-0d45-4bd6-8479-a73b0986eebf None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:16.265413 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e489cb-0d45-4bd6-8479-a73b0986eebf None admin] PATH_INFO: `/v3/projects/516ee04b71684e5885fa9a08b821fb87` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:16.266419 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e489cb-0d45-4bd6-8479-a73b0986eebf None admin] RBAC: Authorizing `identity:get_project(project_id=516ee04b71684e5885fa9a08b821fb87)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:16.268007 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e489cb-0d45-4bd6-8479-a73b0986eebf None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:16.269801 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 66/126] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:16 2022] GET /identity/v3/projects/516ee04b71684e5885fa9a08b821fb87 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:16.275433 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e489cb-0d45-4bd6-8479-a73b0986eebf None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:16.348904 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:16.369412 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 61/127] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:55:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:55:16.370591 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ef5ef5-5818-45a5-949b-c6fce9282f72 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:16.388642 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e489cb-0d45-4bd6-8479-a73b0986eebf None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:16.425350 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cea50a-ac50-4da0-8bf3-0dde2c1241e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:16.426046 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cea50a-ac50-4da0-8bf3-0dde2c1241e2 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:16.426411 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cea50a-ac50-4da0-8bf3-0dde2c1241e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:16.426768 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cea50a-ac50-4da0-8bf3-0dde2c1241e2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:16.439755 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cea50a-ac50-4da0-8bf3-0dde2c1241e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:16.445038 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-68cea50a-ac50-4da0-8bf3-0dde2c1241e2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:55:16.445038 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:16.445038 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:16.445038 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:16.445038 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:16.445038 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:16.445038 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:16.445038 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:16.445038 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:16.445038 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:16.445038 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:16.445038 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:16.445038 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:16.445038 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:16.445038 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:16.445038 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:16.445953 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:16.445953 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:16.445953 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:16.445953 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:16.445953 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:16.445953 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:16.445953 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:16.445953 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:16.445953 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:16.445953 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:16.445953 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:16.445953 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:16.445953 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:16.445953 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:16.445953 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:16.445953 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:16.451539 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:16.451539 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:16.451539 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:16.451539 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:16.451539 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:16.451539 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:16.451539 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:16.451539 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:16.451539 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:16.451539 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:16.451539 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:55:16.451539 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:55:16.451539 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 67/128] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:16.451539 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cea50a-ac50-4da0-8bf3-0dde2c1241e2 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:16.490189 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ef5ef5-5818-45a5-949b-c6fce9282f72 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:16.505728 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:16.530284 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d92994-e42c-4a79-9c43-151f69282afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:16.531212 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d92994-e42c-4a79-9c43-151f69282afb None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:16.531870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d92994-e42c-4a79-9c43-151f69282afb None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:16.532336 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d92994-e42c-4a79-9c43-151f69282afb None admin] PATH_INFO: `/v3/projects/516ee04b71684e5885fa9a08b821fb87` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:16.533814 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d92994-e42c-4a79-9c43-151f69282afb None admin] RBAC: Authorizing `identity:get_project(project_id=516ee04b71684e5885fa9a08b821fb87)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:16.535473 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d92994-e42c-4a79-9c43-151f69282afb None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:16.537624 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 62/129] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:16 2022] GET /identity/v3/projects/516ee04b71684e5885fa9a08b821fb87 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:16.538634 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d92994-e42c-4a79-9c43-151f69282afb None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:16.562068 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cea50a-ac50-4da0-8bf3-0dde2c1241e2 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:16.597608 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683fefd-44c8-4fb3-8425-1389f40fdeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:16.598047 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:16.598458 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683fefd-44c8-4fb3-8425-1389f40fdeca None admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:16.598953 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683fefd-44c8-4fb3-8425-1389f40fdeca None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:16.599076 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683fefd-44c8-4fb3-8425-1389f40fdeca None admin] PATH_INFO: `/v3/projects/516ee04b71684e5885fa9a08b821fb87/users/b3e958a355f0429dab88a0bfd6dd81e6/roles/47e170213d8049a28f8b8a0d736fdf6b` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:16.601755 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9683fefd-44c8-4fb3-8425-1389f40fdeca None admin] RBAC: Authorizing `identity:create_grant(project_id=516ee04b71684e5885fa9a08b821fb87, user_id=b3e958a355f0429dab88a0bfd6dd81e6, role_id=47e170213d8049a28f8b8a0d736fdf6b)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:16.603389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9683fefd-44c8-4fb3-8425-1389f40fdeca None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:16.624690 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 68/130] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 13:55:16 2022] PUT /identity/v3/projects/516ee04b71684e5885fa9a08b821fb87/users/b3e958a355f0429dab88a0bfd6dd81e6/roles/47e170213d8049a28f8b8a0d736fdf6b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:55:16.625818 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683fefd-44c8-4fb3-8425-1389f40fdeca None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:16.657917 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d92994-e42c-4a79-9c43-151f69282afb None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:16.699283 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3e26b-9a9b-44a8-9ac9-6fc5a459f73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:16.700412 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3e26b-9a9b-44a8-9ac9-6fc5a459f73d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:16.700999 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3e26b-9a9b-44a8-9ac9-6fc5a459f73d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:16.701511 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3e26b-9a9b-44a8-9ac9-6fc5a459f73d None admin] PATH_INFO: `/v3/projects/516ee04b71684e5885fa9a08b821fb87` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:16.708274 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc3e26b-9a9b-44a8-9ac9-6fc5a459f73d None admin] RBAC: Authorizing `identity:get_project(project_id=516ee04b71684e5885fa9a08b821fb87)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:16.710127 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc3e26b-9a9b-44a8-9ac9-6fc5a459f73d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:16.712550 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 63/131] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:16 2022] GET /identity/v3/projects/516ee04b71684e5885fa9a08b821fb87 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:16.713608 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3e26b-9a9b-44a8-9ac9-6fc5a459f73d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:16.834875 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683fefd-44c8-4fb3-8425-1389f40fdeca None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:16.852740 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:16.877925 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d1a49-6f38-4042-89f6-fe4c2ce2fd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:16.878556 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d1a49-6f38-4042-89f6-fe4c2ce2fd63 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:16.878839 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d1a49-6f38-4042-89f6-fe4c2ce2fd63 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:16.879111 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d1a49-6f38-4042-89f6-fe4c2ce2fd63 None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:16.879842 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473d1a49-6f38-4042-89f6-fe4c2ce2fd63 None admin] RBAC: Adding query filter params (name=member) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:16.880149 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473d1a49-6f38-4042-89f6-fe4c2ce2fd63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:16.881567 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473d1a49-6f38-4042-89f6-fe4c2ce2fd63 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:16.886712 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3e26b-9a9b-44a8-9ac9-6fc5a459f73d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:16.890984 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 69/132] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 13:55:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:16.895161 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d1a49-6f38-4042-89f6-fe4c2ce2fd63 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:16.909203 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:16.924759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b79710-11ed-4c82-8dfe-c12d532fb60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:16.925453 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b79710-11ed-4c82-8dfe-c12d532fb60e None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:16.925745 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b79710-11ed-4c82-8dfe-c12d532fb60e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:16.926030 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b79710-11ed-4c82-8dfe-c12d532fb60e None admin] PATH_INFO: `/v3/projects/516ee04b71684e5885fa9a08b821fb87/users/b3e958a355f0429dab88a0bfd6dd81e6/roles/926b9f36c24847c582c84af904250abc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:16.928042 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b79710-11ed-4c82-8dfe-c12d532fb60e None admin] RBAC: Authorizing `identity:create_grant(project_id=516ee04b71684e5885fa9a08b821fb87, user_id=b3e958a355f0429dab88a0bfd6dd81e6, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:16.929531 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b79710-11ed-4c82-8dfe-c12d532fb60e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:16.951563 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 64/133] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 13:55:16 2022] PUT /identity/v3/projects/516ee04b71684e5885fa9a08b821fb87/users/b3e958a355f0429dab88a0bfd6dd81e6/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:55:16.956912 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1c111-8c94-43aa-8f6f-a2f4a7067574 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:16.957226 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1c111-8c94-43aa-8f6f-a2f4a7067574 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:16.957515 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1c111-8c94-43aa-8f6f-a2f4a7067574 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:16.959116 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 65/134] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:16.959683 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d1c111-8c94-43aa-8f6f-a2f4a7067574 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:17.022943 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d1a49-6f38-4042-89f6-fe4c2ce2fd63 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:17.064815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f637b96-3b06-4228-99c7-bd74941c4ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:17.065503 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f637b96-3b06-4228-99c7-bd74941c4ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:17.065789 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f637b96-3b06-4228-99c7-bd74941c4ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:17.066070 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f637b96-3b06-4228-99c7-bd74941c4ca7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:17.066925 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f637b96-3b06-4228-99c7-bd74941c4ca7 None admin] RBAC: Adding query filter params (role.id=6617379dc72a414bbf45eae46f9d74b2, scope.project.id=516ee04b71684e5885fa9a08b821fb87, user.id=305f1669ace14628b1c277f969d44fa8) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:17.067242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f637b96-3b06-4228-99c7-bd74941c4ca7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:17.068830 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f637b96-3b06-4228-99c7-bd74941c4ca7 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:17.075522 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 70/135] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:55:16 2022] GET /identity/v3/role_assignments?user.id=305f1669ace14628b1c277f969d44fa8&scope.project.id=516ee04b71684e5885fa9a08b821fb87&role.id=6617379dc72a414bbf45eae46f9d74b2 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:17.076764 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef07ec-1ee8-4ceb-b0ca-c6a43f7c3e2e None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:17.077057 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef07ec-1ee8-4ceb-b0ca-c6a43f7c3e2e None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:17.077339 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef07ec-1ee8-4ceb-b0ca-c6a43f7c3e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:17.133912 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-85ef07ec-1ee8-4ceb-b0ca-c6a43f7c3e2e None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:17.135462 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-85ef07ec-1ee8-4ceb-b0ca-c6a43f7c3e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:17.186478 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d1c111-8c94-43aa-8f6f-a2f4a7067574 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:17.207370 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:17.232909 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077694e5-68d4-4b4e-8dd3-5b1871b63d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:17.233581 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077694e5-68d4-4b4e-8dd3-5b1871b63d4e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:17.233874 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077694e5-68d4-4b4e-8dd3-5b1871b63d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:17.234157 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077694e5-68d4-4b4e-8dd3-5b1871b63d4e None admin] PATH_INFO: `/v3/users/b3e958a355f0429dab88a0bfd6dd81e6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:17.235423 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077694e5-68d4-4b4e-8dd3-5b1871b63d4e None admin] RBAC: Authorizing `identity:get_user(user_id=b3e958a355f0429dab88a0bfd6dd81e6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:17.237070 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077694e5-68d4-4b4e-8dd3-5b1871b63d4e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:17.239856 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 66/136] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:16 2022] GET /identity/v3/users/b3e958a355f0429dab88a0bfd6dd81e6 => generated 275 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:17.251026 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077694e5-68d4-4b4e-8dd3-5b1871b63d4e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:17.294693 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 71/137] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:17.321872 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:17.329623 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ef07ec-1ee8-4ceb-b0ca-c6a43f7c3e2e None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:17.433777 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077694e5-68d4-4b4e-8dd3-5b1871b63d4e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:17.504466 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e95d006-b7c5-4de6-813a-3c2bde22a9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:17.506181 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95d006-b7c5-4de6-813a-3c2bde22a9e0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:17.507701 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95d006-b7c5-4de6-813a-3c2bde22a9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:17.508153 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95d006-b7c5-4de6-813a-3c2bde22a9e0 None admin] PATH_INFO: `/v3/projects/ab213fe217b44205a8e298d3ec240af6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:17.510625 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ef07ec-1ee8-4ceb-b0ca-c6a43f7c3e2e None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:17.513914 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e95d006-b7c5-4de6-813a-3c2bde22a9e0 None admin] RBAC: Authorizing `identity:get_project(project_id=ab213fe217b44205a8e298d3ec240af6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:17.515556 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e95d006-b7c5-4de6-813a-3c2bde22a9e0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:17.518461 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 67/138] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:17 2022] GET /identity/v3/projects/ab213fe217b44205a8e298d3ec240af6 => generated 312 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:17.525044 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e95d006-b7c5-4de6-813a-3c2bde22a9e0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:17.583280 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e3a7be-a602-4a4e-9041-4e80e514b1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:17.584179 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3a7be-a602-4a4e-9041-4e80e514b1fb None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:17.584791 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3a7be-a602-4a4e-9041-4e80e514b1fb None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:17.587171 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3a7be-a602-4a4e-9041-4e80e514b1fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:17.589232 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e3a7be-a602-4a4e-9041-4e80e514b1fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:17.591376 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e3a7be-a602-4a4e-9041-4e80e514b1fb None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:17.593333 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 72/139] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:55:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:17.605411 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e3a7be-a602-4a4e-9041-4e80e514b1fb None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:17.667902 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:17.697475 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e95d006-b7c5-4de6-813a-3c2bde22a9e0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:17.732902 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:17.739797 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e3a7be-a602-4a4e-9041-4e80e514b1fb None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:17.754292 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e36d6-0e33-49b8-8e30-f8d47e0ba020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:17.754995 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e36d6-0e33-49b8-8e30-f8d47e0ba020 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:17.755360 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e36d6-0e33-49b8-8e30-f8d47e0ba020 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:17.755579 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e36d6-0e33-49b8-8e30-f8d47e0ba020 None admin] PATH_INFO: `/v3/projects/ab213fe217b44205a8e298d3ec240af6/users/b3e958a355f0429dab88a0bfd6dd81e6/roles/926b9f36c24847c582c84af904250abc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:17.757604 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1e36d6-0e33-49b8-8e30-f8d47e0ba020 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab213fe217b44205a8e298d3ec240af6, user_id=b3e958a355f0429dab88a0bfd6dd81e6, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:17.759320 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1e36d6-0e33-49b8-8e30-f8d47e0ba020 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:17.788503 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 68/140] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 13:55:17 2022] PUT /identity/v3/projects/ab213fe217b44205a8e298d3ec240af6/users/b3e958a355f0429dab88a0bfd6dd81e6/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:55:17.793097 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fc295a-b5b0-4c7c-aa13-ea4bafc35d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:17.794008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc295a-b5b0-4c7c-aa13-ea4bafc35d87 None admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:17.794458 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc295a-b5b0-4c7c-aa13-ea4bafc35d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:17.794892 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc295a-b5b0-4c7c-aa13-ea4bafc35d87 None admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:17.795756 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fc295a-b5b0-4c7c-aa13-ea4bafc35d87 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:17.800101 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fc295a-b5b0-4c7c-aa13-ea4bafc35d87 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:17.843297 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 73/141] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 13:55:17 2022] POST /identity/v3/users => generated 283 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:55:18.414978 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c03cc8-73da-4e71-a199-c799b86f3425 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:18.415471 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c03cc8-73da-4e71-a199-c799b86f3425 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:18.415567 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c03cc8-73da-4e71-a199-c799b86f3425 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:18.417290 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 69/142] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:18.422935 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2980ef3-2f65-4b74-a9db-c8c0154bbbc8 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:18.423336 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2980ef3-2f65-4b74-a9db-c8c0154bbbc8 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:18.423516 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2980ef3-2f65-4b74-a9db-c8c0154bbbc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:18.455463 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-b2980ef3-2f65-4b74-a9db-c8c0154bbbc8 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:18.456788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-b2980ef3-2f65-4b74-a9db-c8c0154bbbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:18.551943 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 74/143] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:55:18.595066 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c03cc8-73da-4e71-a199-c799b86f3425 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:18.720618 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c03cc8-73da-4e71-a199-c799b86f3425 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:18.768314 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d859af19-087c-4c58-ad57-72f343608be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:18.769029 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859af19-087c-4c58-ad57-72f343608be9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:18.769373 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859af19-087c-4c58-ad57-72f343608be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:18.769666 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859af19-087c-4c58-ad57-72f343608be9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:18.782182 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d859af19-087c-4c58-ad57-72f343608be9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:18.792010 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-d859af19-087c-4c58-ad57-72f343608be9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:55:18.792010 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:18.792010 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:18.792010 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:18.792010 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:18.792010 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:18.792010 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:18.792010 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:18.792010 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:18.792010 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:18.792010 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:18.792010 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:18.792010 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:18.792010 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:18.792010 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:18.792010 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:18.792884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:18.792884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:18.792884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:18.792884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:18.792884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:18.792884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:18.792884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:18.792884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:18.792884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:18.792884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:18.792884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:18.792884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:18.792884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:18.792884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:18.792884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:18.792884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:18.793656 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:18.793656 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:18.793656 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:18.793656 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:18.793656 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:18.793656 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:18.793656 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:18.793656 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:18.793656 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:18.793656 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:18.793656 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:55:18.793656 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:55:18.793656 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 70/144] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 13:55:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:18.800828 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2980ef3-2f65-4b74-a9db-c8c0154bbbc8 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:18.834042 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:18.936381 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2980ef3-2f65-4b74-a9db-c8c0154bbbc8 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:18.978885 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c527a03-14f0-4810-a00b-c62cfdf74130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:18.979452 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c527a03-14f0-4810-a00b-c62cfdf74130 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:18.979740 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c527a03-14f0-4810-a00b-c62cfdf74130 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:18.980039 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c527a03-14f0-4810-a00b-c62cfdf74130 None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:18.980809 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c527a03-14f0-4810-a00b-c62cfdf74130 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:18.981119 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c527a03-14f0-4810-a00b-c62cfdf74130 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:18.982563 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c527a03-14f0-4810-a00b-c62cfdf74130 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:18.993453 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 75/145] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:18.999506 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d859af19-087c-4c58-ad57-72f343608be9 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:19.037780 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:19.116017 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d859af19-087c-4c58-ad57-72f343608be9 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:19.156651 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53388147-19a4-4cae-90a2-7444c7603118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:19.157346 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53388147-19a4-4cae-90a2-7444c7603118 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:19.157656 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53388147-19a4-4cae-90a2-7444c7603118 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:19.158036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53388147-19a4-4cae-90a2-7444c7603118 None admin] PATH_INFO: `/v3/users/7d5a51326470403490a7155c74035d21` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:19.159334 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53388147-19a4-4cae-90a2-7444c7603118 None admin] RBAC: Authorizing `identity:get_user(user_id=7d5a51326470403490a7155c74035d21)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:19.160835 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53388147-19a4-4cae-90a2-7444c7603118 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:19.162454 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 71/146] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:18 2022] GET /identity/v3/users/7d5a51326470403490a7155c74035d21 => generated 247 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:19.168064 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c527a03-14f0-4810-a00b-c62cfdf74130 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:19.279582 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:19.308511 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c527a03-14f0-4810-a00b-c62cfdf74130 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:19.352008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fdf190-6ded-4ac4-9a7d-f4a946562438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:19.353016 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fdf190-6ded-4ac4-9a7d-f4a946562438 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:19.353498 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fdf190-6ded-4ac4-9a7d-f4a946562438 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:19.353986 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fdf190-6ded-4ac4-9a7d-f4a946562438 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:19.432595 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fdf190-6ded-4ac4-9a7d-f4a946562438 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:19.434869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fdf190-6ded-4ac4-9a7d-f4a946562438 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:19.537621 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:19.557618 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 76/147] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:55:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:55:19.563520 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53388147-19a4-4cae-90a2-7444c7603118 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:19.611916 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6527fb-7b35-44ce-befe-1f89b8ea44b7 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:19.612557 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6527fb-7b35-44ce-befe-1f89b8ea44b7 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:19.613053 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6527fb-7b35-44ce-befe-1f89b8ea44b7 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:19.614621 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 77/148] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:19.619950 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b218a-84d7-4580-833a-3f81728c4b6d None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:19.620553 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b218a-84d7-4580-833a-3f81728c4b6d None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:19.621052 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b218a-84d7-4580-833a-3f81728c4b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:19.654209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-b12b218a-84d7-4580-833a-3f81728c4b6d None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:19.655763 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-b12b218a-84d7-4580-833a-3f81728c4b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:19.696789 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53388147-19a4-4cae-90a2-7444c7603118 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:19.708581 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:19.745375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ba72a-f290-4f35-a0d9-77c973016058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:19.746057 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ba72a-f290-4f35-a0d9-77c973016058 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:19.746353 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ba72a-f290-4f35-a0d9-77c973016058 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:19.746642 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ba72a-f290-4f35-a0d9-77c973016058 None admin] PATH_INFO: `/v3/projects/516ee04b71684e5885fa9a08b821fb87` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:19.748415 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ba72a-f290-4f35-a0d9-77c973016058 None admin] RBAC: Authorizing `identity:get_project(project_id=516ee04b71684e5885fa9a08b821fb87)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:19.750110 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ba72a-f290-4f35-a0d9-77c973016058 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:19.752145 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 72/149] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:19 2022] GET /identity/v3/projects/516ee04b71684e5885fa9a08b821fb87 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:19.757628 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ba72a-f290-4f35-a0d9-77c973016058 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:19.781896 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 78/150] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:55:19.819266 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b218a-84d7-4580-833a-3f81728c4b6d None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:19.951876 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ba72a-f290-4f35-a0d9-77c973016058 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:19.979049 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b218a-84d7-4580-833a-3f81728c4b6d None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:20.010283 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:20.038344 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e41d602-c4f9-457e-a6ef-03203f189791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:20.039290 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41d602-c4f9-457e-a6ef-03203f189791 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:20.039573 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41d602-c4f9-457e-a6ef-03203f189791 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:20.039854 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41d602-c4f9-457e-a6ef-03203f189791 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:20.053750 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90959fea-e9b9-41cc-9f28-f2ae3198a825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:20.054370 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90959fea-e9b9-41cc-9f28-f2ae3198a825 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:20.054652 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90959fea-e9b9-41cc-9f28-f2ae3198a825 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:20.054931 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90959fea-e9b9-41cc-9f28-f2ae3198a825 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:20.055735 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90959fea-e9b9-41cc-9f28-f2ae3198a825 None admin] RBAC: Adding query filter params (role.id=58f91e3a13664dabb18c6227a1658fa6, scope.project.id=516ee04b71684e5885fa9a08b821fb87, user.id=7d5a51326470403490a7155c74035d21) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:20.056042 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90959fea-e9b9-41cc-9f28-f2ae3198a825 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:20.057374 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e41d602-c4f9-457e-a6ef-03203f189791 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:20.057500 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90959fea-e9b9-41cc-9f28-f2ae3198a825 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:20.062893 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-7e41d602-c4f9-457e-a6ef-03203f189791 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:55:20.062893 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:20.062893 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:20.062893 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:20.062893 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:20.062893 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:20.062893 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:20.062893 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:20.062893 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:20.062893 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:20.062893 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:20.062893 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:20.062893 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:20.062893 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:20.062893 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:20.062893 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:20.063786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:20.063786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:20.063786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:20.063786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:20.063786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:20.063786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:20.063786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:20.063786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:20.063786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:20.063786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:20.063786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:20.063786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:20.063786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:20.063786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:20.063786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:20.063786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:20.064590 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:20.064590 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:20.064590 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:20.064590 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:20.064590 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:20.064590 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:20.064590 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:20.064590 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:20.064590 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:20.064590 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:20.064590 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:55:20.064590 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:55:20.064590 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 79/151] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 13:55:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:20.065740 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfae9d-ea53-4d35-a3f8-f947f6ac3e91 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:20.065740 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfae9d-ea53-4d35-a3f8-f947f6ac3e91 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:20.065920 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfae9d-ea53-4d35-a3f8-f947f6ac3e91 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:20.067307 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 80/152] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:20.073061 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cfae9d-ea53-4d35-a3f8-f947f6ac3e91 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:20.074807 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 73/153] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:55:19 2022] GET /identity/v3/role_assignments?user.id=7d5a51326470403490a7155c74035d21&scope.project.id=516ee04b71684e5885fa9a08b821fb87&role.id=58f91e3a13664dabb18c6227a1658fa6 => generated 608 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:20.076432 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37b2a2-5f9b-4869-bd87-b81abaacf287 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:20.080852 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37b2a2-5f9b-4869-bd87-b81abaacf287 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:20.081160 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37b2a2-5f9b-4869-bd87-b81abaacf287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:20.121508 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-9f37b2a2-5f9b-4869-bd87-b81abaacf287 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:20.123422 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9f37b2a2-5f9b-4869-bd87-b81abaacf287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:20.207834 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:20.237192 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cfae9d-ea53-4d35-a3f8-f947f6ac3e91 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:20.288061 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a1818c-c2f4-4b07-bab7-914075681482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:20.288712 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1818c-c2f4-4b07-bab7-914075681482 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:20.288998 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1818c-c2f4-4b07-bab7-914075681482 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:20.289278 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1818c-c2f4-4b07-bab7-914075681482 None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:20.290018 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a1818c-c2f4-4b07-bab7-914075681482 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:20.290519 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a1818c-c2f4-4b07-bab7-914075681482 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:20.291966 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a1818c-c2f4-4b07-bab7-914075681482 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:20.306191 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 81/154] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:55:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:20.312656 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a1818c-c2f4-4b07-bab7-914075681482 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:20.363274 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:20.377881 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 74/155] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:20.422969 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37b2a2-5f9b-4869-bd87-b81abaacf287 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:20.448014 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a1818c-c2f4-4b07-bab7-914075681482 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:20.518657 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed4c8ad-8183-4b98-84a2-37b2dd26a428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:20.519324 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4c8ad-8183-4b98-84a2-37b2dd26a428 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:20.519610 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4c8ad-8183-4b98-84a2-37b2dd26a428 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:20.519891 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4c8ad-8183-4b98-84a2-37b2dd26a428 None admin] PATH_INFO: `/v3/users/b3e958a355f0429dab88a0bfd6dd81e6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:20.525234 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed4c8ad-8183-4b98-84a2-37b2dd26a428 None admin] RBAC: Authorizing `identity:get_user(user_id=b3e958a355f0429dab88a0bfd6dd81e6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:20.526711 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed4c8ad-8183-4b98-84a2-37b2dd26a428 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:20.529114 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 82/156] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:20 2022] GET /identity/v3/users/b3e958a355f0429dab88a0bfd6dd81e6 => generated 275 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:20.534264 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed4c8ad-8183-4b98-84a2-37b2dd26a428 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:20.572730 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37b2a2-5f9b-4869-bd87-b81abaacf287 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:20.630185 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb669a50-cc21-47cd-b2f1-ad729d895538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:20.630770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb669a50-cc21-47cd-b2f1-ad729d895538 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:20.631052 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb669a50-cc21-47cd-b2f1-ad729d895538 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:20.631327 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb669a50-cc21-47cd-b2f1-ad729d895538 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:20.649875 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb669a50-cc21-47cd-b2f1-ad729d895538 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:20.655380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-fb669a50-cc21-47cd-b2f1-ad729d895538 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 13:55:20.655380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:20.655380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:20.655380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:20.655380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:20.655380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:20.655380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:20.655380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:20.655380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:20.655380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:20.655380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:20.655380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:20.655380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:20.655380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:20.655380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:20.655380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:20.656511 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:20.656511 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:20.656511 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:20.656511 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:20.656511 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:20.656511 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:20.656511 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:20.656511 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:20.656511 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:20.656511 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:20.656511 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:20.656511 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:20.656511 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:20.656511 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:20.656511 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:20.656511 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:20.657382 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:20.657382 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:20.657382 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:20.657382 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:20.657382 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:20.657382 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:20.657382 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:20.657382 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:20.657382 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:20.657382 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:20.657382 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 13:55:20.657382 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:55:20.660973 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 75/157] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:20.666841 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb669a50-cc21-47cd-b2f1-ad729d895538 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:20.709375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:20.721550 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed4c8ad-8183-4b98-84a2-37b2dd26a428 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:20.728839 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:20.789215 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437cb738-82d1-4fa3-a95b-8a71a0559832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:20.789865 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437cb738-82d1-4fa3-a95b-8a71a0559832 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:20.790157 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437cb738-82d1-4fa3-a95b-8a71a0559832 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:20.790441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437cb738-82d1-4fa3-a95b-8a71a0559832 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:20.828413 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb669a50-cc21-47cd-b2f1-ad729d895538 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:20.885449 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b466a-5553-4451-9c85-8ee35cb1a138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:20.886027 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b466a-5553-4451-9c85-8ee35cb1a138 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:20.886280 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b466a-5553-4451-9c85-8ee35cb1a138 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:20.886556 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b466a-5553-4451-9c85-8ee35cb1a138 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:20.887838 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b466a-5553-4451-9c85-8ee35cb1a138 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:20.888149 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b466a-5553-4451-9c85-8ee35cb1a138 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:20.889619 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b466a-5553-4451-9c85-8ee35cb1a138 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:20.892681 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437cb738-82d1-4fa3-a95b-8a71a0559832 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:20.894195 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437cb738-82d1-4fa3-a95b-8a71a0559832 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:20.899224 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 76/158] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 13:55:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:20.900337 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691cb71-69fd-40f2-91e0-bad81c81a4fb None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:20.900765 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691cb71-69fd-40f2-91e0-bad81c81a4fb None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:20.900936 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691cb71-69fd-40f2-91e0-bad81c81a4fb None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:20.902276 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 77/159] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:20.906799 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9691cb71-69fd-40f2-91e0-bad81c81a4fb None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:21.067012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 83/160] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:55:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 13:55:21.067406 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1ee51-7025-459a-9f18-c56798ab86e0 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:21.067597 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1ee51-7025-459a-9f18-c56798ab86e0 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:21.067834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1ee51-7025-459a-9f18-c56798ab86e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:21.068423 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:21.089067 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:21.091619 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9691cb71-69fd-40f2-91e0-bad81c81a4fb None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:21.109514 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-ade1ee51-7025-459a-9f18-c56798ab86e0 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:21.110730 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-ade1ee51-7025-459a-9f18-c56798ab86e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:21.156434 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33443d8-c35b-4657-8120-5ce5f99913a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:21.157122 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33443d8-c35b-4657-8120-5ce5f99913a7 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:21.157406 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33443d8-c35b-4657-8120-5ce5f99913a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:21.157683 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33443d8-c35b-4657-8120-5ce5f99913a7 None admin] PATH_INFO: `/v3/users/305f1669ace14628b1c277f969d44fa8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:21.161189 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33443d8-c35b-4657-8120-5ce5f99913a7 None admin] RBAC: Authorizing `identity:get_user(user_id=305f1669ace14628b1c277f969d44fa8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:21.162634 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33443d8-c35b-4657-8120-5ce5f99913a7 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:21.164245 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 78/161] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:20 2022] GET /identity/v3/users/305f1669ace14628b1c277f969d44fa8 => generated 289 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:21.165075 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33443d8-c35b-4657-8120-5ce5f99913a7 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:21.250007 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 84/162] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:21.250708 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade1ee51-7025-459a-9f18-c56798ab86e0 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:21.331258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33443d8-c35b-4657-8120-5ce5f99913a7 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:21.410058 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676696a-f855-4171-afac-f68ec02f3b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:21.410946 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676696a-f855-4171-afac-f68ec02f3b00 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:21.411240 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676696a-f855-4171-afac-f68ec02f3b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:21.411520 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676696a-f855-4171-afac-f68ec02f3b00 None admin] PATH_INFO: `/v3/projects/516ee04b71684e5885fa9a08b821fb87` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:21.416502 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9676696a-f855-4171-afac-f68ec02f3b00 None admin] RBAC: Authorizing `identity:get_project(project_id=516ee04b71684e5885fa9a08b821fb87)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:21.416616 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9676696a-f855-4171-afac-f68ec02f3b00 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:21.419171 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 79/163] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:21 2022] GET /identity/v3/projects/516ee04b71684e5885fa9a08b821fb87 => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:21.419938 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676696a-f855-4171-afac-f68ec02f3b00 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:21.442661 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade1ee51-7025-459a-9f18-c56798ab86e0 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:21.470856 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:21.520581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f681ed-ee13-4084-bcff-ddbaf94f9bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:21.521569 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f681ed-ee13-4084-bcff-ddbaf94f9bcd None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:21.521843 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f681ed-ee13-4084-bcff-ddbaf94f9bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:21.522222 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f681ed-ee13-4084-bcff-ddbaf94f9bcd None admin] PATH_INFO: `/v3/projects/516ee04b71684e5885fa9a08b821fb87` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:21.523495 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f681ed-ee13-4084-bcff-ddbaf94f9bcd None admin] RBAC: Authorizing `identity:get_project(project_id=516ee04b71684e5885fa9a08b821fb87)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:21.532123 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f681ed-ee13-4084-bcff-ddbaf94f9bcd None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:21.534312 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 85/164] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:21 2022] GET /identity/v3/projects/516ee04b71684e5885fa9a08b821fb87 => generated 298 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:21.535192 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f681ed-ee13-4084-bcff-ddbaf94f9bcd None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:21.542813 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:21.561761 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676696a-f855-4171-afac-f68ec02f3b00 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:21.600793 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ef8c6-c278-48f0-af2f-439a634b285b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:21.602218 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ef8c6-c278-48f0-af2f-439a634b285b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:21.602664 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ef8c6-c278-48f0-af2f-439a634b285b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:21.603082 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ef8c6-c278-48f0-af2f-439a634b285b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:21.615482 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5ef8c6-c278-48f0-af2f-439a634b285b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:21.621166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-5a5ef8c6-c278-48f0-af2f-439a634b285b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:55:21.621166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:21.621166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:21.621166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:21.621166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:21.621166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:21.621166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:21.621166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:21.621166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:21.621166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:21.621166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:21.621166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:21.621166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:21.621166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:21.621166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:21.621166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:21.622365 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:21.622365 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:21.622365 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:21.622365 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:21.622365 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:21.622365 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:21.622365 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:21.622365 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:21.622365 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:21.622365 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:21.622365 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:21.622365 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:21.622365 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:21.622365 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:21.622365 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:21.622365 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:21.623279 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:21.623279 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:21.623279 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:21.623279 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:21.623279 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:21.623279 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:21.623279 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:21.623279 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:21.623279 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:21.623279 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:21.623279 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:55:21.623279 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:55:21.631452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 80/165] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:21.633042 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf5b40-4b22-4aad-b5f1-fa0e16649c57 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:21.634245 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf5b40-4b22-4aad-b5f1-fa0e16649c57 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:21.634726 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf5b40-4b22-4aad-b5f1-fa0e16649c57 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:21.637786 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 81/166] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:21.638640 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf5b40-4b22-4aad-b5f1-fa0e16649c57 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:21.740081 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:21.759798 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf5b40-4b22-4aad-b5f1-fa0e16649c57 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:21.806213 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e5551-17fc-43a1-af27-4e4979b599ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:21.807162 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e5551-17fc-43a1-af27-4e4979b599ee None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:21.807585 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e5551-17fc-43a1-af27-4e4979b599ee None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:21.807998 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e5551-17fc-43a1-af27-4e4979b599ee None admin] PATH_INFO: `/v3/projects/516ee04b71684e5885fa9a08b821fb87/users/305f1669ace14628b1c277f969d44fa8/roles/6617379dc72a414bbf45eae46f9d74b2` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:21.812417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40e5551-17fc-43a1-af27-4e4979b599ee None admin] RBAC: Authorizing `identity:create_grant(project_id=516ee04b71684e5885fa9a08b821fb87, user_id=305f1669ace14628b1c277f969d44fa8, role_id=6617379dc72a414bbf45eae46f9d74b2)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:21.814059 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40e5551-17fc-43a1-af27-4e4979b599ee None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:21.823608 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f681ed-ee13-4084-bcff-ddbaf94f9bcd None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:21.847304 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 82/167] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 13:55:21 2022] PUT /identity/v3/projects/516ee04b71684e5885fa9a08b821fb87/users/305f1669ace14628b1c277f969d44fa8/roles/6617379dc72a414bbf45eae46f9d74b2 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:55:21.848423 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e5551-17fc-43a1-af27-4e4979b599ee None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:21.867678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe00bdd-f820-4edd-912f-52633edab45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:21.868765 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe00bdd-f820-4edd-912f-52633edab45c None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:21.869235 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe00bdd-f820-4edd-912f-52633edab45c None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:21.869742 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe00bdd-f820-4edd-912f-52633edab45c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:21.870756 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe00bdd-f820-4edd-912f-52633edab45c None admin] RBAC: Adding query filter params (role.id=47e170213d8049a28f8b8a0d736fdf6b, scope.project.id=516ee04b71684e5885fa9a08b821fb87, user.id=b3e958a355f0429dab88a0bfd6dd81e6) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:21.871325 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe00bdd-f820-4edd-912f-52633edab45c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:21.874185 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe00bdd-f820-4edd-912f-52633edab45c None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:21.890829 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 86/168] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:55:21 2022] GET /identity/v3/role_assignments?user.id=b3e958a355f0429dab88a0bfd6dd81e6&scope.project.id=516ee04b71684e5885fa9a08b821fb87&role.id=47e170213d8049a28f8b8a0d736fdf6b => generated 608 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:21.892421 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c0d42-c576-4068-9049-2a5fcaddda3c None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:21.898951 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c0d42-c576-4068-9049-2a5fcaddda3c None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:21.900080 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c0d42-c576-4068-9049-2a5fcaddda3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:21.947831 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-c65c0d42-c576-4068-9049-2a5fcaddda3c None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:21.953611 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-c65c0d42-c576-4068-9049-2a5fcaddda3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:22.092982 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:22.096390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 87/169] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:22.128590 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e5551-17fc-43a1-af27-4e4979b599ee None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:22.141591 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c0d42-c576-4068-9049-2a5fcaddda3c None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:22.192954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71407c8-33cd-4176-92d3-202b9a0bb7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:22.193597 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71407c8-33cd-4176-92d3-202b9a0bb7f4 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:22.193880 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71407c8-33cd-4176-92d3-202b9a0bb7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:22.194156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71407c8-33cd-4176-92d3-202b9a0bb7f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:22.194890 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71407c8-33cd-4176-92d3-202b9a0bb7f4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:22.195195 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71407c8-33cd-4176-92d3-202b9a0bb7f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:22.196624 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71407c8-33cd-4176-92d3-202b9a0bb7f4 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:22.210375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 83/170] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 13:55:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:22.218540 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71407c8-33cd-4176-92d3-202b9a0bb7f4 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:22.229417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:22.337651 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c0d42-c576-4068-9049-2a5fcaddda3c None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:22.388217 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6eff7e-81b6-4e93-9d5c-c7badc2ec388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:22.388955 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6eff7e-81b6-4e93-9d5c-c7badc2ec388 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:22.389243 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6eff7e-81b6-4e93-9d5c-c7badc2ec388 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:22.389522 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6eff7e-81b6-4e93-9d5c-c7badc2ec388 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:22.401709 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6eff7e-81b6-4e93-9d5c-c7badc2ec388 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:22.407041 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-8d6eff7e-81b6-4e93-9d5c-c7badc2ec388 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:55:22.407041 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:22.407041 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:22.407041 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:22.407041 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:22.407041 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:22.407041 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:22.407041 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:22.407041 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:22.407041 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:22.407041 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:22.407041 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:22.407041 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:22.407041 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:22.407041 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:22.407041 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:22.407898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:22.407898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:22.407898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:22.407898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:22.407898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:22.407898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:22.407898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:22.407898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:22.407898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:22.407898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:22.407898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:22.407898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:22.407898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:22.407898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:22.407898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:22.407898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:22.408678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:22.408678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:22.408678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:22.408678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:22.408678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:22.408678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:22.408678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:22.408678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:22.408678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:22.408678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:22.408678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:55:22.408678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:55:22.408678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 88/171] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:55:22.410902 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105d429-ac04-4002-bb71-f578a5bc941d None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:22.411275 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105d429-ac04-4002-bb71-f578a5bc941d None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:22.411540 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105d429-ac04-4002-bb71-f578a5bc941d None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:22.412884 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 89/172] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:22.417682 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3105d429-ac04-4002-bb71-f578a5bc941d None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:22.435154 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71407c8-33cd-4176-92d3-202b9a0bb7f4 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:22.478330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc104984-cbad-4a92-bdf5-eeba09a629a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:22.478988 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc104984-cbad-4a92-bdf5-eeba09a629a1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:22.479273 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc104984-cbad-4a92-bdf5-eeba09a629a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:22.479552 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc104984-cbad-4a92-bdf5-eeba09a629a1 None admin] PATH_INFO: `/v3/users/b3e958a355f0429dab88a0bfd6dd81e6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:22.480807 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc104984-cbad-4a92-bdf5-eeba09a629a1 None admin] RBAC: Authorizing `identity:get_user(user_id=b3e958a355f0429dab88a0bfd6dd81e6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:22.482849 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc104984-cbad-4a92-bdf5-eeba09a629a1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:22.488643 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 84/173] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:22 2022] GET /identity/v3/users/b3e958a355f0429dab88a0bfd6dd81e6 => generated 275 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:22.489951 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c3ad22-a72d-4144-9e04-86bd1338a9f2 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:22.490235 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c3ad22-a72d-4144-9e04-86bd1338a9f2 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:22.490515 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c3ad22-a72d-4144-9e04-86bd1338a9f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:22.512367 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c8c3ad22-a72d-4144-9e04-86bd1338a9f2 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:22.513030 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:22.513816 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c8c3ad22-a72d-4144-9e04-86bd1338a9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:22.579995 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3105d429-ac04-4002-bb71-f578a5bc941d None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:22.607729 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 85/174] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:22.608643 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c3ad22-a72d-4144-9e04-86bd1338a9f2 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:22.616474 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:22.647878 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693581a2-c8cc-4ada-a873-a949555f497a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:22.648557 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693581a2-c8cc-4ada-a873-a949555f497a None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:22.648846 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693581a2-c8cc-4ada-a873-a949555f497a None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:22.649125 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693581a2-c8cc-4ada-a873-a949555f497a None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:22.649861 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693581a2-c8cc-4ada-a873-a949555f497a None admin] RBAC: Adding query filter params (name=member) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:22.650168 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693581a2-c8cc-4ada-a873-a949555f497a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:22.654137 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693581a2-c8cc-4ada-a873-a949555f497a None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:22.671564 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 90/175] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 13:55:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:22.672458 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693581a2-c8cc-4ada-a873-a949555f497a None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:22.756627 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c3ad22-a72d-4144-9e04-86bd1338a9f2 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:22.805516 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c90dd97-4e85-4362-9f5e-8994afc4da07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:22.806150 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90dd97-4e85-4362-9f5e-8994afc4da07 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:22.806431 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90dd97-4e85-4362-9f5e-8994afc4da07 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:22.806710 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90dd97-4e85-4362-9f5e-8994afc4da07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:22.807835 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693581a2-c8cc-4ada-a873-a949555f497a None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:22.828098 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:22.847709 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d16e2-d783-44e5-9a60-8b4a8e56938b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:22.848365 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d16e2-d783-44e5-9a60-8b4a8e56938b None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:22.848679 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d16e2-d783-44e5-9a60-8b4a8e56938b None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:22.848983 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d16e2-d783-44e5-9a60-8b4a8e56938b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:22.859962 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d16e2-d783-44e5-9a60-8b4a8e56938b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:22.865512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-165d16e2-d783-44e5-9a60-8b4a8e56938b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:55:22.865512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:22.865512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:22.865512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:22.865512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:22.865512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:22.865512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:22.865512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:22.865512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:22.865512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:22.865512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:22.865512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:22.865512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:22.865512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:22.865512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:22.865512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:22.866359 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:22.866359 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:22.866359 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:22.866359 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:22.866359 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:22.866359 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:22.866359 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:22.866359 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:22.866359 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:22.866359 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:22.866359 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:22.866359 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:22.866359 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:22.866359 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:22.866359 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:22.866359 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:22.867131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:22.867131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:22.867131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:22.867131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:22.867131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:22.867131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:22.867131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:22.867131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:22.867131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:22.867131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:22.867131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:55:22.867131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:55:22.867131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 91/176] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 13:55:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:22.867766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe83f5-6d1c-490a-ab3c-09d98e4690cd None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:22.868315 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe83f5-6d1c-490a-ab3c-09d98e4690cd None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:22.868315 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe83f5-6d1c-490a-ab3c-09d98e4690cd None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:22.869487 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 92/177] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:22.873690 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe83f5-6d1c-490a-ab3c-09d98e4690cd None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:22.884108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c90dd97-4e85-4362-9f5e-8994afc4da07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:22.885588 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c90dd97-4e85-4362-9f5e-8994afc4da07 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:22.889161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:22.981717 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe83f5-6d1c-490a-ab3c-09d98e4690cd None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:23.007050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 86/178] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:55:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:55:23.008655 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb844b3-2757-4907-89fe-ab2356badc55 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:23.009292 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb844b3-2757-4907-89fe-ab2356badc55 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:23.009671 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb844b3-2757-4907-89fe-ab2356badc55 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:23.011032 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 87/179] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:23.015630 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb844b3-2757-4907-89fe-ab2356badc55 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:23.022439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb901c7-f923-4cef-a482-f584f4161b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:23.023083 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb901c7-f923-4cef-a482-f584f4161b3f None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:23.023369 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb901c7-f923-4cef-a482-f584f4161b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:23.023651 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb901c7-f923-4cef-a482-f584f4161b3f None admin] PATH_INFO: `/v3/users/b3e958a355f0429dab88a0bfd6dd81e6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:23.024883 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb901c7-f923-4cef-a482-f584f4161b3f None admin] RBAC: Authorizing `identity:get_user(user_id=b3e958a355f0429dab88a0bfd6dd81e6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:23.026345 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb901c7-f923-4cef-a482-f584f4161b3f None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:23.030991 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 93/180] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:22 2022] GET /identity/v3/users/b3e958a355f0429dab88a0bfd6dd81e6 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:23.030991 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9956d0-6300-48a1-b8e1-95726d54ffc8 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:23.030991 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9956d0-6300-48a1-b8e1-95726d54ffc8 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:23.030991 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9956d0-6300-48a1-b8e1-95726d54ffc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:23.048170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-df9956d0-6300-48a1-b8e1-95726d54ffc8 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:23.049529 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-df9956d0-6300-48a1-b8e1-95726d54ffc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:23.094921 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:23.124454 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 94/181] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:23.129478 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9956d0-6300-48a1-b8e1-95726d54ffc8 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:23.129616 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb844b3-2757-4907-89fe-ab2356badc55 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:23.132708 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:23.169846 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7704af7b-549e-4b38-b2a5-b83c5325d94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:23.170574 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7704af7b-549e-4b38-b2a5-b83c5325d94f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:23.170968 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7704af7b-549e-4b38-b2a5-b83c5325d94f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:23.171250 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7704af7b-549e-4b38-b2a5-b83c5325d94f None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:23.171968 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7704af7b-549e-4b38-b2a5-b83c5325d94f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:23.172276 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7704af7b-549e-4b38-b2a5-b83c5325d94f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:23.173676 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7704af7b-549e-4b38-b2a5-b83c5325d94f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:23.182825 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 88/182] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:23.186309 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b4649-6072-487c-af86-daaf150378ec None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:23.186923 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b4649-6072-487c-af86-daaf150378ec None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:23.187541 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b4649-6072-487c-af86-daaf150378ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:23.207862 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4e1b4649-6072-487c-af86-daaf150378ec None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:23.209086 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4e1b4649-6072-487c-af86-daaf150378ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:23.239225 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9956d0-6300-48a1-b8e1-95726d54ffc8 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:23.275826 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d5dc1-e92b-4d43-939c-e55c2f38fdcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:23.276565 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d5dc1-e92b-4d43-939c-e55c2f38fdcf None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:23.276886 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d5dc1-e92b-4d43-939c-e55c2f38fdcf None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:23.277120 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d5dc1-e92b-4d43-939c-e55c2f38fdcf None admin] PATH_INFO: `/v3/projects/516ee04b71684e5885fa9a08b821fb87` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:23.278897 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d5dc1-e92b-4d43-939c-e55c2f38fdcf None admin] RBAC: Authorizing `identity:get_project(project_id=516ee04b71684e5885fa9a08b821fb87)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:23.286802 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 89/183] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:23.286938 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d5dc1-e92b-4d43-939c-e55c2f38fdcf None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:23.287011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b4649-6072-487c-af86-daaf150378ec None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:23.287077 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 95/184] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:23 2022] GET /identity/v3/projects/516ee04b71684e5885fa9a08b821fb87 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:23.287077 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d5dc1-e92b-4d43-939c-e55c2f38fdcf None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:23.338769 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:23.394667 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:23.402052 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b4649-6072-487c-af86-daaf150378ec None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:23.402306 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d5dc1-e92b-4d43-939c-e55c2f38fdcf None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:23.439385 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65775ee-4ea2-4843-8e19-595c418da415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:23.439740 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ce55d-239d-44d2-a2f2-50540b673d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:23.440036 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65775ee-4ea2-4843-8e19-595c418da415 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:23.440319 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65775ee-4ea2-4843-8e19-595c418da415 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:23.440415 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ce55d-239d-44d2-a2f2-50540b673d17 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:23.440650 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ce55d-239d-44d2-a2f2-50540b673d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:23.440745 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65775ee-4ea2-4843-8e19-595c418da415 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:23.440925 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ce55d-239d-44d2-a2f2-50540b673d17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:23.452668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65775ee-4ea2-4843-8e19-595c418da415 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:23.457571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-a65775ee-4ea2-4843-8e19-595c418da415 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:55:23.457571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:23.457571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:23.457571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:23.457571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:23.457571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:23.457571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:23.457571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:23.457571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:23.457571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:23.457571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:23.457571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:23.457571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:23.457571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:23.457571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:23.457571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:23.458479 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:23.458479 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:23.458479 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:23.458479 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:23.458479 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:23.458479 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:23.458479 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:23.458479 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:23.458479 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:23.458479 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:23.458479 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:23.458479 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:23.458479 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:23.458479 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:23.458479 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:23.458479 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:23.459249 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:23.459249 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:23.459249 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:23.459249 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:23.459249 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:23.459249 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:23.459249 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:23.459249 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:23.459249 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:23.459249 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:23.459249 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:55:23.459249 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:55:23.459249 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 96/185] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 13:55:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:23.459871 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65775ee-4ea2-4843-8e19-595c418da415 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:23.523042 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605ce55d-239d-44d2-a2f2-50540b673d17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:23.524426 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605ce55d-239d-44d2-a2f2-50540b673d17 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:23.567958 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65775ee-4ea2-4843-8e19-595c418da415 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:23.571175 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:23.605149 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf8d1e-fddb-4510-87e8-334f61642f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:23.606486 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf8d1e-fddb-4510-87e8-334f61642f9e None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:23.606764 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf8d1e-fddb-4510-87e8-334f61642f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:23.607034 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf8d1e-fddb-4510-87e8-334f61642f9e None admin] PATH_INFO: `/v3/users/a61d02fa4d1c4567a197ea1027e1ffdc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:23.620850 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cf8d1e-fddb-4510-87e8-334f61642f9e None admin] RBAC: Authorizing `identity:get_user(user_id=a61d02fa4d1c4567a197ea1027e1ffdc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:23.622833 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cf8d1e-fddb-4510-87e8-334f61642f9e None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:23.624792 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:23.629317 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 97/186] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:23 2022] GET /identity/v3/users/a61d02fa4d1c4567a197ea1027e1ffdc => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:23.630158 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf8d1e-fddb-4510-87e8-334f61642f9e None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:23.632539 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 90/187] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:55:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:55:23.633288 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ce55d-239d-44d2-a2f2-50540b673d17 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:23.738239 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ce55d-239d-44d2-a2f2-50540b673d17 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:23.739850 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf8d1e-fddb-4510-87e8-334f61642f9e None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:23.775685 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c2c2e-dd6f-44c1-8167-535fce542885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:23.776247 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c2c2e-dd6f-44c1-8167-535fce542885 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:23.776548 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c2c2e-dd6f-44c1-8167-535fce542885 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:23.776825 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c2c2e-dd6f-44c1-8167-535fce542885 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:23.778836 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36c2c2e-dd6f-44c1-8167-535fce542885 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:23.780327 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36c2c2e-dd6f-44c1-8167-535fce542885 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:23.783069 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 91/188] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:55:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:23.784027 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4f21c-2000-485f-b34b-f06828a051b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:23.784892 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4f21c-2000-485f-b34b-f06828a051b4 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:23.785329 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c2c2e-dd6f-44c1-8167-535fce542885 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:23.785605 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4f21c-2000-485f-b34b-f06828a051b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:23.786176 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4f21c-2000-485f-b34b-f06828a051b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:23.787168 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da4f21c-2000-485f-b34b-f06828a051b4 None admin] RBAC: Adding query filter params (role.id=926b9f36c24847c582c84af904250abc, scope.project.id=516ee04b71684e5885fa9a08b821fb87, user.id=b3e958a355f0429dab88a0bfd6dd81e6) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:23.787683 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da4f21c-2000-485f-b34b-f06828a051b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:23.789482 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da4f21c-2000-485f-b34b-f06828a051b4 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:23.796467 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 98/189] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:55:23 2022] GET /identity/v3/role_assignments?user.id=b3e958a355f0429dab88a0bfd6dd81e6&scope.project.id=516ee04b71684e5885fa9a08b821fb87&role.id=926b9f36c24847c582c84af904250abc => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:23.801094 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4f21c-2000-485f-b34b-f06828a051b4 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:23.837580 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:23.873213 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:23.922945 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c2c2e-dd6f-44c1-8167-535fce542885 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:23.943949 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4f21c-2000-485f-b34b-f06828a051b4 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:23.976148 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e2d8c7-3ffb-4216-855a-9e23d7faf9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:23.976954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2d8c7-3ffb-4216-855a-9e23d7faf9dd None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:23.977334 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2d8c7-3ffb-4216-855a-9e23d7faf9dd None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:23.977922 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2d8c7-3ffb-4216-855a-9e23d7faf9dd None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:23.978537 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e2d8c7-3ffb-4216-855a-9e23d7faf9dd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:23.978981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e2d8c7-3ffb-4216-855a-9e23d7faf9dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:23.980575 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e2d8c7-3ffb-4216-855a-9e23d7faf9dd None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:23.988695 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6432e17-95e4-405d-9b94-273f0e73abed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:23.989531 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6432e17-95e4-405d-9b94-273f0e73abed None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:23.989825 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6432e17-95e4-405d-9b94-273f0e73abed None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:23.990102 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6432e17-95e4-405d-9b94-273f0e73abed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:23.990754 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 92/190] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:55:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:23.992282 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e2d8c7-3ffb-4216-855a-9e23d7faf9dd None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:24.085341 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6432e17-95e4-405d-9b94-273f0e73abed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:24.086778 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6432e17-95e4-405d-9b94-273f0e73abed None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:24.122744 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:24.124448 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e2d8c7-3ffb-4216-855a-9e23d7faf9dd None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:24.167192 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:24.179581 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca30f1d-7585-4839-85b0-d4750e6adaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:24.180237 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca30f1d-7585-4839-85b0-d4750e6adaa5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:24.180578 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca30f1d-7585-4839-85b0-d4750e6adaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:24.180862 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca30f1d-7585-4839-85b0-d4750e6adaa5 None admin] PATH_INFO: `/v3/projects/ab213fe217b44205a8e298d3ec240af6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:24.182165 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca30f1d-7585-4839-85b0-d4750e6adaa5 None admin] RBAC: Authorizing `identity:get_project(project_id=ab213fe217b44205a8e298d3ec240af6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:24.183732 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca30f1d-7585-4839-85b0-d4750e6adaa5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:24.185808 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 93/191] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:23 2022] GET /identity/v3/projects/ab213fe217b44205a8e298d3ec240af6 => generated 312 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:24.191037 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca30f1d-7585-4839-85b0-d4750e6adaa5 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:24.208515 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 99/192] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:55:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:55:24.208515 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6432e17-95e4-405d-9b94-273f0e73abed None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:24.303327 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca30f1d-7585-4839-85b0-d4750e6adaa5 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:24.307966 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6432e17-95e4-405d-9b94-273f0e73abed None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:24.332572 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:24.337987 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2cfd95-a3f8-4197-986e-b453e117234a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:24.338559 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2cfd95-a3f8-4197-986e-b453e117234a None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:24.338846 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2cfd95-a3f8-4197-986e-b453e117234a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:24.339186 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2cfd95-a3f8-4197-986e-b453e117234a None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:24.339880 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2cfd95-a3f8-4197-986e-b453e117234a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:24.341858 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2cfd95-a3f8-4197-986e-b453e117234a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:24.342636 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1defb307-b0ed-4747-b53c-6cb172fe5e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:24.343376 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1defb307-b0ed-4747-b53c-6cb172fe5e16 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:24.343666 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1defb307-b0ed-4747-b53c-6cb172fe5e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:24.343943 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1defb307-b0ed-4747-b53c-6cb172fe5e16 None admin] PATH_INFO: `/v3/users/a61d02fa4d1c4567a197ea1027e1ffdc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:24.345117 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1defb307-b0ed-4747-b53c-6cb172fe5e16 None admin] RBAC: Authorizing `identity:get_user(user_id=a61d02fa4d1c4567a197ea1027e1ffdc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:24.346545 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1defb307-b0ed-4747-b53c-6cb172fe5e16 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:24.348298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 100/193] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:24 2022] GET /identity/v3/users/a61d02fa4d1c4567a197ea1027e1ffdc => generated 283 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:24.348908 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1defb307-b0ed-4747-b53c-6cb172fe5e16 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:24.367656 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 94/194] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 13:55:24 2022] POST /identity/v3/users => generated 297 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:55:24.369842 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2cfd95-a3f8-4197-986e-b453e117234a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:24.428759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:24.466064 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1defb307-b0ed-4747-b53c-6cb172fe5e16 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:24.495039 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2cfd95-a3f8-4197-986e-b453e117234a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:24.517031 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b6b0d7-9029-48cf-95b1-0a53f65d4ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:24.519054 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6b0d7-9029-48cf-95b1-0a53f65d4ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:24.519054 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6b0d7-9029-48cf-95b1-0a53f65d4ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:24.519054 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6b0d7-9029-48cf-95b1-0a53f65d4ea6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:24.519054 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b6b0d7-9029-48cf-95b1-0a53f65d4ea6 None admin] RBAC: Adding query filter params (role.id=926b9f36c24847c582c84af904250abc, scope.project.id=ab213fe217b44205a8e298d3ec240af6, user.id=b3e958a355f0429dab88a0bfd6dd81e6) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:24.519328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b6b0d7-9029-48cf-95b1-0a53f65d4ea6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:24.520746 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b6b0d7-9029-48cf-95b1-0a53f65d4ea6 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:24.528437 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 101/195] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:55:24 2022] GET /identity/v3/role_assignments?user.id=b3e958a355f0429dab88a0bfd6dd81e6&scope.project.id=ab213fe217b44205a8e298d3ec240af6&role.id=926b9f36c24847c582c84af904250abc => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:24.529922 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc97c0-184f-4cb9-9e95-dba7021ed148 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:24.530233 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc97c0-184f-4cb9-9e95-dba7021ed148 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:24.530511 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc97c0-184f-4cb9-9e95-dba7021ed148 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:24.531726 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 102/196] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:24.532423 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fc97c0-184f-4cb9-9e95-dba7021ed148 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:24.554888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472cba5-3ed4-4b8f-a557-c7c62053a9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:24.555792 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472cba5-3ed4-4b8f-a557-c7c62053a9d4 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:24.556199 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472cba5-3ed4-4b8f-a557-c7c62053a9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:24.556669 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472cba5-3ed4-4b8f-a557-c7c62053a9d4 None admin] PATH_INFO: `/v3/projects/05e772d8318b47469008ecb063004d71` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:24.558852 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472cba5-3ed4-4b8f-a557-c7c62053a9d4 None admin] RBAC: Authorizing `identity:get_project(project_id=05e772d8318b47469008ecb063004d71)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:24.560387 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472cba5-3ed4-4b8f-a557-c7c62053a9d4 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:24.562416 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 95/197] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:24 2022] GET /identity/v3/projects/05e772d8318b47469008ecb063004d71 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:24.565859 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6bff87-5e4b-47c0-9365-34142ac5ee46 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:24.567812 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6bff87-5e4b-47c0-9365-34142ac5ee46 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:24.568200 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6bff87-5e4b-47c0-9365-34142ac5ee46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:24.594619 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-7b6bff87-5e4b-47c0-9365-34142ac5ee46 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:24.596718 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7b6bff87-5e4b-47c0-9365-34142ac5ee46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:24.619413 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:24.705832 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fc97c0-184f-4cb9-9e95-dba7021ed148 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:24.706308 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 96/198] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:24.707356 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6bff87-5e4b-47c0-9365-34142ac5ee46 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:24.751275 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608e9ee-dbf8-487e-a0d6-1995799ccc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:24.752191 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608e9ee-dbf8-487e-a0d6-1995799ccc38 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:24.752687 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608e9ee-dbf8-487e-a0d6-1995799ccc38 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:24.753106 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608e9ee-dbf8-487e-a0d6-1995799ccc38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:24.771995 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:24.832848 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0608e9ee-dbf8-487e-a0d6-1995799ccc38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:24.834387 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0608e9ee-dbf8-487e-a0d6-1995799ccc38 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:24.841081 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6bff87-5e4b-47c0-9365-34142ac5ee46 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:24.888957 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf974f85-725d-485d-b1f1-cb9e84fcee25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:24.890045 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf974f85-725d-485d-b1f1-cb9e84fcee25 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:24.890611 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf974f85-725d-485d-b1f1-cb9e84fcee25 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:24.891081 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf974f85-725d-485d-b1f1-cb9e84fcee25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:24.892212 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf974f85-725d-485d-b1f1-cb9e84fcee25 None admin] RBAC: Adding query filter params (role.id=58f91e3a13664dabb18c6227a1658fa6, scope.project.id=05e772d8318b47469008ecb063004d71, user.id=a61d02fa4d1c4567a197ea1027e1ffdc) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:24.892743 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf974f85-725d-485d-b1f1-cb9e84fcee25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:24.894680 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf974f85-725d-485d-b1f1-cb9e84fcee25 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:24.901527 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 97/199] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:55:24 2022] GET /identity/v3/role_assignments?user.id=a61d02fa4d1c4567a197ea1027e1ffdc&scope.project.id=05e772d8318b47469008ecb063004d71&role.id=58f91e3a13664dabb18c6227a1658fa6 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:24.903839 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf974f85-725d-485d-b1f1-cb9e84fcee25 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:24.904995 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:24.958037 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 103/200] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:55:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:55:24.963532 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608e9ee-dbf8-487e-a0d6-1995799ccc38 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:25.039608 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf974f85-725d-485d-b1f1-cb9e84fcee25 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:25.057323 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:25.083843 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d83ec8-60fc-4da3-bd37-1272ac06a975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:25.084779 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d83ec8-60fc-4da3-bd37-1272ac06a975 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:25.085159 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d83ec8-60fc-4da3-bd37-1272ac06a975 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:25.085553 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d83ec8-60fc-4da3-bd37-1272ac06a975 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:25.104473 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d83ec8-60fc-4da3-bd37-1272ac06a975 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:25.114436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-42d83ec8-60fc-4da3-bd37-1272ac06a975 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 13:55:25.114436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:25.114436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:25.114436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:25.114436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:25.114436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:25.114436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:25.114436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:25.114436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:25.114436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:25.114436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:25.114436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:25.114436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:25.114436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:25.114436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:25.114436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:25.115544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:25.115544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:25.115544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:25.115544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:25.115544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:25.115544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:25.115544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:25.115544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:25.115544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:25.115544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:25.115544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:25.115544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:25.115544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:25.115544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:25.115544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:25.115544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:25.116543 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:25.116543 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:25.116543 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:25.116543 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:25.116543 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:25.116543 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:25.116543 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:25.116543 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:25.116543 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:25.116543 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:25.116543 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 13:55:25.116543 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:55:25.119690 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 98/201] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:25.133516 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d83ec8-60fc-4da3-bd37-1272ac06a975 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:25.161691 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608e9ee-dbf8-487e-a0d6-1995799ccc38 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:25.201176 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef85a25-34c7-4a80-9080-f920b8de9379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:25.202017 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef85a25-34c7-4a80-9080-f920b8de9379 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:25.202374 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef85a25-34c7-4a80-9080-f920b8de9379 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:25.202759 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef85a25-34c7-4a80-9080-f920b8de9379 None admin] PATH_INFO: `/v3/projects/05e772d8318b47469008ecb063004d71` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:25.204067 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef85a25-34c7-4a80-9080-f920b8de9379 None admin] RBAC: Authorizing `identity:get_project(project_id=05e772d8318b47469008ecb063004d71)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:25.205704 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef85a25-34c7-4a80-9080-f920b8de9379 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:25.207441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 104/202] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:24 2022] GET /identity/v3/projects/05e772d8318b47469008ecb063004d71 => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:25.219346 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef85a25-34c7-4a80-9080-f920b8de9379 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:25.265186 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:25.328132 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d83ec8-60fc-4da3-bd37-1272ac06a975 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:25.348876 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef85a25-34c7-4a80-9080-f920b8de9379 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:25.386367 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2dede6-a630-48c4-ab49-091432d5be20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:25.387518 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2dede6-a630-48c4-ab49-091432d5be20 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:25.387803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2dede6-a630-48c4-ab49-091432d5be20 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:25.388077 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2dede6-a630-48c4-ab49-091432d5be20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:25.388905 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2dede6-a630-48c4-ab49-091432d5be20 None admin] RBAC: Adding query filter params (role.id=47e170213d8049a28f8b8a0d736fdf6b, scope.project.id=05e772d8318b47469008ecb063004d71, user.id=a61d02fa4d1c4567a197ea1027e1ffdc) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:25.389247 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2dede6-a630-48c4-ab49-091432d5be20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:25.390775 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2dede6-a630-48c4-ab49-091432d5be20 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:25.396672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 105/203] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:55:25 2022] GET /identity/v3/role_assignments?user.id=a61d02fa4d1c4567a197ea1027e1ffdc&scope.project.id=05e772d8318b47469008ecb063004d71&role.id=47e170213d8049a28f8b8a0d736fdf6b => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:25.401968 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f16dfb-e254-4bcb-be60-d86b7637f5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:25.403167 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f16dfb-e254-4bcb-be60-d86b7637f5f7 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:25.403509 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f16dfb-e254-4bcb-be60-d86b7637f5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:25.403796 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f16dfb-e254-4bcb-be60-d86b7637f5f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:25.404619 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f16dfb-e254-4bcb-be60-d86b7637f5f7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:25.404936 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f16dfb-e254-4bcb-be60-d86b7637f5f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:25.406413 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f16dfb-e254-4bcb-be60-d86b7637f5f7 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:25.427964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 99/204] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 13:55:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:25.433480 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2dede6-a630-48c4-ab49-091432d5be20 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:25.544353 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:25.556337 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2dede6-a630-48c4-ab49-091432d5be20 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:25.594262 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e586a5e6-acd8-4e98-bf4a-c0ac0b378aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:25.594915 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586a5e6-acd8-4e98-bf4a-c0ac0b378aba None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:25.595195 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586a5e6-acd8-4e98-bf4a-c0ac0b378aba None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:25.595471 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586a5e6-acd8-4e98-bf4a-c0ac0b378aba None admin] PATH_INFO: `/v3/users/305f1669ace14628b1c277f969d44fa8` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:25.596706 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e586a5e6-acd8-4e98-bf4a-c0ac0b378aba None admin] RBAC: Authorizing `identity:get_user(user_id=305f1669ace14628b1c277f969d44fa8)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:25.598147 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e586a5e6-acd8-4e98-bf4a-c0ac0b378aba None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:25.600155 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 106/205] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:25 2022] GET /identity/v3/users/305f1669ace14628b1c277f969d44fa8 => generated 289 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:25.605429 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f16dfb-e254-4bcb-be60-d86b7637f5f7 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:25.647273 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:25.729133 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f16dfb-e254-4bcb-be60-d86b7637f5f7 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:25.784108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e179bd5b-0913-40a8-a4f8-a3b4b1df33fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:25.785140 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179bd5b-0913-40a8-a4f8-a3b4b1df33fe None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:25.785617 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179bd5b-0913-40a8-a4f8-a3b4b1df33fe None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:25.786079 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179bd5b-0913-40a8-a4f8-a3b4b1df33fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:25.859270 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e179bd5b-0913-40a8-a4f8-a3b4b1df33fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:25.861561 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e179bd5b-0913-40a8-a4f8-a3b4b1df33fe None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:25.895794 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:25.988616 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 100/206] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:55:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:55:25.993998 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e586a5e6-acd8-4e98-bf4a-c0ac0b378aba None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:26.117292 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e586a5e6-acd8-4e98-bf4a-c0ac0b378aba None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:26.168450 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fecf0f7-87a7-4620-9b13-4a9e421165d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:26.169148 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fecf0f7-87a7-4620-9b13-4a9e421165d8 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:26.169437 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fecf0f7-87a7-4620-9b13-4a9e421165d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:26.169721 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fecf0f7-87a7-4620-9b13-4a9e421165d8 None admin] PATH_INFO: `/v3/projects/516ee04b71684e5885fa9a08b821fb87` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:26.170907 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fecf0f7-87a7-4620-9b13-4a9e421165d8 None admin] RBAC: Authorizing `identity:get_project(project_id=516ee04b71684e5885fa9a08b821fb87)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:26.172363 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fecf0f7-87a7-4620-9b13-4a9e421165d8 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:26.174475 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 107/207] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:25 2022] GET /identity/v3/projects/516ee04b71684e5885fa9a08b821fb87 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:26.179905 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e179bd5b-0913-40a8-a4f8-a3b4b1df33fe None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:26.323578 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e179bd5b-0913-40a8-a4f8-a3b4b1df33fe None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:26.362552 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a39e1-bca8-481e-a37d-26ba8e6cefb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:26.363544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a39e1-bca8-481e-a37d-26ba8e6cefb3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:26.364032 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a39e1-bca8-481e-a37d-26ba8e6cefb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:26.364539 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a39e1-bca8-481e-a37d-26ba8e6cefb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:26.365535 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566a39e1-bca8-481e-a37d-26ba8e6cefb3 None admin] RBAC: Adding query filter params (role.id=6617379dc72a414bbf45eae46f9d74b2, scope.project.id=516ee04b71684e5885fa9a08b821fb87, user.id=305f1669ace14628b1c277f969d44fa8) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:26.366068 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566a39e1-bca8-481e-a37d-26ba8e6cefb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:26.367722 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566a39e1-bca8-481e-a37d-26ba8e6cefb3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:26.369824 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:26.375116 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 101/208] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:55:26 2022] GET /identity/v3/role_assignments?user.id=305f1669ace14628b1c277f969d44fa8&scope.project.id=516ee04b71684e5885fa9a08b821fb87&role.id=6617379dc72a414bbf45eae46f9d74b2 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:27.016579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c823a45-6100-4d80-a5d5-071181fe949f None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:27.017092 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c823a45-6100-4d80-a5d5-071181fe949f None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:27.017182 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c823a45-6100-4d80-a5d5-071181fe949f None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:27.018786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 108/209] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:27.024709 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572d9e6-15eb-40f5-bce8-2856cb8eae6d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:27.025110 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572d9e6-15eb-40f5-bce8-2856cb8eae6d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:27.025298 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572d9e6-15eb-40f5-bce8-2856cb8eae6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:27.047937 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4572d9e6-15eb-40f5-bce8-2856cb8eae6d None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:27.049166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4572d9e6-15eb-40f5-bce8-2856cb8eae6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:27.132666 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 102/210] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:27.150791 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c823a45-6100-4d80-a5d5-071181fe949f None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:27.180759 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:27.259942 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c823a45-6100-4d80-a5d5-071181fe949f None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:27.262639 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e356f0-0447-4267-9b73-dd1697ed5ac3 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:27.262930 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e356f0-0447-4267-9b73-dd1697ed5ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:27.263209 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e356f0-0447-4267-9b73-dd1697ed5ac3 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:27.265005 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 103/211] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:27.269900 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb4f33-4ece-47f0-9d5e-d145481576cc None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:27.270188 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb4f33-4ece-47f0-9d5e-d145481576cc None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:27.270471 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb4f33-4ece-47f0-9d5e-d145481576cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:27.288193 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-04eb4f33-4ece-47f0-9d5e-d145481576cc None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:27.289427 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-04eb4f33-4ece-47f0-9d5e-d145481576cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:27.298762 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4734d32-c3e9-4dd4-ae6c-bbe27b1d98af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:27.299795 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4734d32-c3e9-4dd4-ae6c-bbe27b1d98af None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:27.300086 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4734d32-c3e9-4dd4-ae6c-bbe27b1d98af None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:27.300369 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4734d32-c3e9-4dd4-ae6c-bbe27b1d98af None admin] PATH_INFO: `/v3/roles/member` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:27.315383 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4734d32-c3e9-4dd4-ae6c-bbe27b1d98af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:27.327172 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-d4734d32-c3e9-4dd4-ae6c-bbe27b1d98af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:55:27.327172 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:27.327172 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:27.327172 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:27.327172 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:27.327172 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:27.327172 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:27.327172 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:27.327172 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:27.327172 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:27.327172 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:27.327172 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:27.327172 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:27.327172 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:27.327172 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:27.327172 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:27.328410 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:27.328410 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:27.328410 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:27.328410 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:27.328410 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:27.328410 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:27.328410 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:27.328410 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:27.328410 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:27.328410 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:27.328410 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:27.328410 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:27.328410 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:27.328410 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:27.328410 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:27.328410 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:27.329426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:27.329426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:27.329426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:27.329426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:27.329426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:27.329426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:27.329426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:27.329426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:27.329426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:27.329426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:27.329426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:55:27.329426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:55:27.332604 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 109/212] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:27.336059 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4734d32-c3e9-4dd4-ae6c-bbe27b1d98af None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:27.362836 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 104/213] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:27.377142 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc53fb-8e23-4c7c-b418-f3aee9331301 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:27.377759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc53fb-8e23-4c7c-b418-f3aee9331301 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:27.378250 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc53fb-8e23-4c7c-b418-f3aee9331301 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:27.380096 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 105/214] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:27.390453 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc53fb-8e23-4c7c-b418-f3aee9331301 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:27.439807 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:27.452564 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:27.456679 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4734d32-c3e9-4dd4-ae6c-bbe27b1d98af None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:27.490993 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158ed60-bb3f-4388-95c2-9e681dc703c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:27.491610 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158ed60-bb3f-4388-95c2-9e681dc703c8 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:27.491891 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158ed60-bb3f-4388-95c2-9e681dc703c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:27.492337 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158ed60-bb3f-4388-95c2-9e681dc703c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:27.493104 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9158ed60-bb3f-4388-95c2-9e681dc703c8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:27.493412 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9158ed60-bb3f-4388-95c2-9e681dc703c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:27.494780 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9158ed60-bb3f-4388-95c2-9e681dc703c8 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:27.497000 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc53fb-8e23-4c7c-b418-f3aee9331301 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:27.503671 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 110/215] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 13:55:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:27.505784 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b2f34-0c61-4f44-9fd3-ea006bf05181 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:27.506083 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b2f34-0c61-4f44-9fd3-ea006bf05181 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:27.506348 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b2f34-0c61-4f44-9fd3-ea006bf05181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:27.528938 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-cb5b2f34-0c61-4f44-9fd3-ea006bf05181 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:27.530129 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-cb5b2f34-0c61-4f44-9fd3-ea006bf05181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:27.537331 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692054c-ae90-403e-89b8-f35e7d935398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:27.537982 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692054c-ae90-403e-89b8-f35e7d935398 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:27.538267 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692054c-ae90-403e-89b8-f35e7d935398 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:27.538546 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692054c-ae90-403e-89b8-f35e7d935398 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:27.539778 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a692054c-ae90-403e-89b8-f35e7d935398 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:27.541150 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a692054c-ae90-403e-89b8-f35e7d935398 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:27.542948 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 106/216] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:55:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:27.547935 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692054c-ae90-403e-89b8-f35e7d935398 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:27.600562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 111/217] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:27.600562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b2f34-0c61-4f44-9fd3-ea006bf05181 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:27.657396 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692054c-ae90-403e-89b8-f35e7d935398 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:27.683405 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:27.692548 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:27.693547 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f862ca-32cb-491d-b29c-f86a016fbf54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:27.694328 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f862ca-32cb-491d-b29c-f86a016fbf54 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:27.694665 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f862ca-32cb-491d-b29c-f86a016fbf54 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:27.694941 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f862ca-32cb-491d-b29c-f86a016fbf54 None admin] PATH_INFO: `/v3/users/f9c077f97eba4ff98d11f234d5ecc446` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:27.709833 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f862ca-32cb-491d-b29c-f86a016fbf54 None admin] RBAC: Authorizing `identity:get_user(user_id=f9c077f97eba4ff98d11f234d5ecc446)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:27.711293 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f862ca-32cb-491d-b29c-f86a016fbf54 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:27.712944 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 107/218] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:27 2022] GET /identity/v3/users/f9c077f97eba4ff98d11f234d5ecc446 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:27.713707 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f862ca-32cb-491d-b29c-f86a016fbf54 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:27.714228 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b2f34-0c61-4f44-9fd3-ea006bf05181 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:27.749915 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41bcf14-a219-4736-8332-275e083043a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:27.750535 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41bcf14-a219-4736-8332-275e083043a7 None admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:27.750817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41bcf14-a219-4736-8332-275e083043a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:27.751094 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41bcf14-a219-4736-8332-275e083043a7 None admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:27.751769 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41bcf14-a219-4736-8332-275e083043a7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:27.753230 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41bcf14-a219-4736-8332-275e083043a7 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:27.775846 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 112/219] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 13:55:27 2022] POST /identity/v3/users => generated 297 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:55:27.777113 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7104ad9-8428-4c60-87d6-086e7e9550dc None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:27.777405 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7104ad9-8428-4c60-87d6-086e7e9550dc None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:27.777684 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7104ad9-8428-4c60-87d6-086e7e9550dc None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:27.778926 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 113/220] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:27.779643 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7104ad9-8428-4c60-87d6-086e7e9550dc None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:27.840932 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f862ca-32cb-491d-b29c-f86a016fbf54 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:27.877893 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6930c50-4994-4da9-a963-fe890353e9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:27.878668 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6930c50-4994-4da9-a963-fe890353e9da None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:27.879004 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6930c50-4994-4da9-a963-fe890353e9da None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:27.879291 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6930c50-4994-4da9-a963-fe890353e9da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:27.893301 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6930c50-4994-4da9-a963-fe890353e9da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:27.898622 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-e6930c50-4994-4da9-a963-fe890353e9da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:55:27.898622 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:27.898622 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:27.898622 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:27.898622 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:27.898622 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:27.898622 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:27.898622 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:27.898622 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:27.898622 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:27.898622 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:27.898622 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:27.898622 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:27.898622 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:27.898622 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:27.898622 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:27.899559 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:27.899559 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:27.899559 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:27.899559 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:27.899559 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:27.899559 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:27.899559 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:27.899559 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:27.899559 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:27.899559 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:27.899559 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:27.899559 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:27.899559 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:27.899559 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:27.899559 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:27.899559 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:27.900330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:27.900330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:27.900330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:27.900330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:27.900330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:27.900330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:27.900330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:27.900330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:27.900330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:27.900330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:27.900330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:55:27.900330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:55:27.900330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 108/221] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 13:55:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:27.901548 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46014b1a-c972-4bd7-a503-a17aeeda4574 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:27.901645 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46014b1a-c972-4bd7-a503-a17aeeda4574 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:27.901902 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46014b1a-c972-4bd7-a503-a17aeeda4574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:27.917057 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7104ad9-8428-4c60-87d6-086e7e9550dc None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:27.922521 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-46014b1a-c972-4bd7-a503-a17aeeda4574 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:27.923752 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-46014b1a-c972-4bd7-a503-a17aeeda4574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:27.968061 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6342b68a-f486-4f4b-b19b-b901766c21b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:27.968860 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342b68a-f486-4f4b-b19b-b901766c21b3 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:27.969144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342b68a-f486-4f4b-b19b-b901766c21b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:27.970161 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342b68a-f486-4f4b-b19b-b901766c21b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:27.972512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:27.975431 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:28.019953 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 109/222] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:28.020221 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46014b1a-c972-4bd7-a503-a17aeeda4574 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:28.059321 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6342b68a-f486-4f4b-b19b-b901766c21b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:28.061126 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6342b68a-f486-4f4b-b19b-b901766c21b3 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:28.142644 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46014b1a-c972-4bd7-a503-a17aeeda4574 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:28.187218 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe02ab2-7cff-4d8e-9a3b-ab5cc1739def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:28.187774 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe02ab2-7cff-4d8e-9a3b-ab5cc1739def None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:28.188066 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe02ab2-7cff-4d8e-9a3b-ab5cc1739def None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:28.188351 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe02ab2-7cff-4d8e-9a3b-ab5cc1739def None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:28.189187 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 114/223] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:55:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:55:28.189426 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe02ab2-7cff-4d8e-9a3b-ab5cc1739def None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:28.189727 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe02ab2-7cff-4d8e-9a3b-ab5cc1739def None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:28.190026 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6342b68a-f486-4f4b-b19b-b901766c21b3 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:28.191269 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe02ab2-7cff-4d8e-9a3b-ab5cc1739def None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:28.201195 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 110/224] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:28.201862 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe02ab2-7cff-4d8e-9a3b-ab5cc1739def None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:28.235549 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:28.266716 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:28.338333 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6342b68a-f486-4f4b-b19b-b901766c21b3 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:28.340910 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe02ab2-7cff-4d8e-9a3b-ab5cc1739def None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:28.377562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f22bb4-2dea-4056-92be-9e65993444c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:28.378310 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f22bb4-2dea-4056-92be-9e65993444c5 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:28.378817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f22bb4-2dea-4056-92be-9e65993444c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:28.379159 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f22bb4-2dea-4056-92be-9e65993444c5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:28.385257 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29948f8-4700-48d5-b238-11ab7e5ce074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:28.386027 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29948f8-4700-48d5-b238-11ab7e5ce074 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:28.386320 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29948f8-4700-48d5-b238-11ab7e5ce074 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:28.387108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29948f8-4700-48d5-b238-11ab7e5ce074 None admin] PATH_INFO: `/v3/projects/05e772d8318b47469008ecb063004d71` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:28.388606 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29948f8-4700-48d5-b238-11ab7e5ce074 None admin] RBAC: Authorizing `identity:get_project(project_id=05e772d8318b47469008ecb063004d71)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:28.390561 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29948f8-4700-48d5-b238-11ab7e5ce074 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:28.391876 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f22bb4-2dea-4056-92be-9e65993444c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:28.392870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 111/225] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:28 2022] GET /identity/v3/projects/05e772d8318b47469008ecb063004d71 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:28.393664 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29948f8-4700-48d5-b238-11ab7e5ce074 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:28.403533 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-55f22bb4-2dea-4056-92be-9e65993444c5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:55:28.403533 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:28.403533 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:28.403533 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:28.403533 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:28.403533 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:28.403533 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:28.403533 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:28.403533 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:28.403533 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:28.403533 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:28.403533 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:28.403533 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:28.403533 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:28.403533 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:28.403533 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:28.404425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:28.404425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:28.404425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:28.404425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:28.404425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:28.404425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:28.404425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:28.404425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:28.404425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:28.404425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:28.404425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:28.404425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:28.404425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:28.404425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:28.404425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:28.404425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:28.405209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:28.405209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:28.405209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:28.405209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:28.405209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:28.405209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:28.405209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:28.405209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:28.405209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:28.405209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:28.405209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:55:28.405209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:55:28.405907 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 115/226] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 13:55:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:28.406844 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f22bb4-2dea-4056-92be-9e65993444c5 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:28.524450 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:28.525385 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29948f8-4700-48d5-b238-11ab7e5ce074 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:28.529559 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:28.535457 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f22bb4-2dea-4056-92be-9e65993444c5 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:28.573787 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72cae14-b968-429d-a990-62667cf3696a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:28.574616 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad17792-538a-421b-8c18-6e9a73800293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:28.575117 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cae14-b968-429d-a990-62667cf3696a None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:28.575457 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad17792-538a-421b-8c18-6e9a73800293 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:28.575557 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cae14-b968-429d-a990-62667cf3696a None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:28.575848 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cae14-b968-429d-a990-62667cf3696a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:28.575954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad17792-538a-421b-8c18-6e9a73800293 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:28.576255 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad17792-538a-421b-8c18-6e9a73800293 None admin] PATH_INFO: `/v3/users/a61d02fa4d1c4567a197ea1027e1ffdc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:28.576914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72cae14-b968-429d-a990-62667cf3696a None admin] RBAC: Adding query filter params (role.id=926b9f36c24847c582c84af904250abc, scope.project.id=05e772d8318b47469008ecb063004d71, user.id=f9c077f97eba4ff98d11f234d5ecc446) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:28.577233 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72cae14-b968-429d-a990-62667cf3696a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:28.577710 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad17792-538a-421b-8c18-6e9a73800293 None admin] RBAC: Authorizing `identity:get_user(user_id=a61d02fa4d1c4567a197ea1027e1ffdc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:28.578797 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72cae14-b968-429d-a990-62667cf3696a None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:28.579395 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad17792-538a-421b-8c18-6e9a73800293 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:28.581615 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 112/227] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:28 2022] GET /identity/v3/users/a61d02fa4d1c4567a197ea1027e1ffdc => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:28.582261 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad17792-538a-421b-8c18-6e9a73800293 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:28.590425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 116/228] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:55:28 2022] GET /identity/v3/role_assignments?user.id=f9c077f97eba4ff98d11f234d5ecc446&scope.project.id=05e772d8318b47469008ecb063004d71&role.id=926b9f36c24847c582c84af904250abc => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:28.591280 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72cae14-b968-429d-a990-62667cf3696a None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:28.747746 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72cae14-b968-429d-a990-62667cf3696a None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:28.774902 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad17792-538a-421b-8c18-6e9a73800293 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:28.792349 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d8304c-fe01-4c2a-b694-85f19c3b55d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:28.793106 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8304c-fe01-4c2a-b694-85f19c3b55d5 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:28.793388 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8304c-fe01-4c2a-b694-85f19c3b55d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:28.793668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8304c-fe01-4c2a-b694-85f19c3b55d5 None admin] PATH_INFO: `/v3/projects/05e772d8318b47469008ecb063004d71` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:28.794894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d8304c-fe01-4c2a-b694-85f19c3b55d5 None admin] RBAC: Authorizing `identity:get_project(project_id=05e772d8318b47469008ecb063004d71)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:28.796377 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d8304c-fe01-4c2a-b694-85f19c3b55d5 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:28.804261 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 117/229] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:28 2022] GET /identity/v3/projects/05e772d8318b47469008ecb063004d71 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:28.810070 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d8304c-fe01-4c2a-b694-85f19c3b55d5 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:28.842958 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:28.866569 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b96c57b-180b-44de-9d3b-6935029840fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:28.867209 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96c57b-180b-44de-9d3b-6935029840fa None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:28.867518 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96c57b-180b-44de-9d3b-6935029840fa None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:28.867925 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96c57b-180b-44de-9d3b-6935029840fa None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:28.868694 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b96c57b-180b-44de-9d3b-6935029840fa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:28.869367 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b96c57b-180b-44de-9d3b-6935029840fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:28.871612 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b96c57b-180b-44de-9d3b-6935029840fa None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:28.885114 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 113/230] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:55:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:28.892438 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b96c57b-180b-44de-9d3b-6935029840fa None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:28.960881 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:28.973865 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d8304c-fe01-4c2a-b694-85f19c3b55d5 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:29.026112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9e521-8858-47b3-893c-79dd76e5630e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:29.026730 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9e521-8858-47b3-893c-79dd76e5630e None admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:29.027021 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9e521-8858-47b3-893c-79dd76e5630e None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:29.027341 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9e521-8858-47b3-893c-79dd76e5630e None admin] PATH_INFO: `/v3/projects/05e772d8318b47469008ecb063004d71/users/a61d02fa4d1c4567a197ea1027e1ffdc/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:29.032877 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff9e521-8858-47b3-893c-79dd76e5630e None admin] RBAC: Authorizing `identity:create_grant(project_id=05e772d8318b47469008ecb063004d71, user_id=a61d02fa4d1c4567a197ea1027e1ffdc, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:29.034754 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff9e521-8858-47b3-893c-79dd76e5630e None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:29.060362 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 118/231] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 13:55:28 2022] PUT /identity/v3/projects/05e772d8318b47469008ecb063004d71/users/a61d02fa4d1c4567a197ea1027e1ffdc/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:55:29.082527 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b96c57b-180b-44de-9d3b-6935029840fa None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:29.132690 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ca82d-bb0d-4f4e-b359-ded132774a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:29.133990 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ca82d-bb0d-4f4e-b359-ded132774a98 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:29.134476 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ca82d-bb0d-4f4e-b359-ded132774a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:29.134975 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ca82d-bb0d-4f4e-b359-ded132774a98 None admin] PATH_INFO: `/v3/users/a61d02fa4d1c4567a197ea1027e1ffdc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:29.136708 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351ca82d-bb0d-4f4e-b359-ded132774a98 None admin] RBAC: Authorizing `identity:get_user(user_id=a61d02fa4d1c4567a197ea1027e1ffdc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:29.138563 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351ca82d-bb0d-4f4e-b359-ded132774a98 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:29.140762 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 114/232] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:28 2022] GET /identity/v3/users/a61d02fa4d1c4567a197ea1027e1ffdc => generated 283 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:29.149778 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9e521-8858-47b3-893c-79dd76e5630e None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:29.270412 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:29.288896 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9e521-8858-47b3-893c-79dd76e5630e None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:29.329284 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44cdd23-e52d-4548-aa28-81fcc56369af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:29.330450 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44cdd23-e52d-4548-aa28-81fcc56369af None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:29.330945 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44cdd23-e52d-4548-aa28-81fcc56369af None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:29.331427 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44cdd23-e52d-4548-aa28-81fcc56369af None admin] PATH_INFO: `/v3/projects/05e772d8318b47469008ecb063004d71` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:29.333095 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44cdd23-e52d-4548-aa28-81fcc56369af None admin] RBAC: Authorizing `identity:get_project(project_id=05e772d8318b47469008ecb063004d71)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:29.334799 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44cdd23-e52d-4548-aa28-81fcc56369af None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:29.336812 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 119/233] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:29 2022] GET /identity/v3/projects/05e772d8318b47469008ecb063004d71 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:29.343703 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ca82d-bb0d-4f4e-b359-ded132774a98 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:29.474253 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ca82d-bb0d-4f4e-b359-ded132774a98 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:29.476913 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:29.518827 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7acbc-cd5d-4967-a46a-c4cf1e9be37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:29.521213 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7acbc-cd5d-4967-a46a-c4cf1e9be37c None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:29.521725 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7acbc-cd5d-4967-a46a-c4cf1e9be37c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:29.522205 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7acbc-cd5d-4967-a46a-c4cf1e9be37c None admin] PATH_INFO: `/v3/projects/05e772d8318b47469008ecb063004d71/users/a61d02fa4d1c4567a197ea1027e1ffdc/roles/47e170213d8049a28f8b8a0d736fdf6b` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:29.524703 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab7acbc-cd5d-4967-a46a-c4cf1e9be37c None admin] RBAC: Authorizing `identity:create_grant(project_id=05e772d8318b47469008ecb063004d71, user_id=a61d02fa4d1c4567a197ea1027e1ffdc, role_id=47e170213d8049a28f8b8a0d736fdf6b)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:29.526361 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab7acbc-cd5d-4967-a46a-c4cf1e9be37c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:29.547605 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 115/234] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 13:55:29 2022] PUT /identity/v3/projects/05e772d8318b47469008ecb063004d71/users/a61d02fa4d1c4567a197ea1027e1ffdc/roles/47e170213d8049a28f8b8a0d736fdf6b => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:55:30.085472 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c3eb9-4cca-4ae8-b4dc-0da048816f7f None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:30.086414 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c3eb9-4cca-4ae8-b4dc-0da048816f7f None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:30.086925 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c3eb9-4cca-4ae8-b4dc-0da048816f7f None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:30.088753 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 120/235] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:30.093692 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f28472-21c2-4748-b8b0-e689a10335b1 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:30.094212 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f28472-21c2-4748-b8b0-e689a10335b1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:30.094704 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f28472-21c2-4748-b8b0-e689a10335b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:30.116180 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d4f28472-21c2-4748-b8b0-e689a10335b1 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:30.117707 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d4f28472-21c2-4748-b8b0-e689a10335b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:30.161150 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe7792-a42c-4d92-9db7-9fa3db3552f6 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:30.161828 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe7792-a42c-4d92-9db7-9fa3db3552f6 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:30.162363 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe7792-a42c-4d92-9db7-9fa3db3552f6 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:30.163981 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 121/236] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:30.169580 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47ad37-ed33-425f-bcc8-da017a1933d5 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:30.170108 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47ad37-ed33-425f-bcc8-da017a1933d5 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:30.170584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47ad37-ed33-425f-bcc8-da017a1933d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:30.198395 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-fe47ad37-ed33-425f-bcc8-da017a1933d5 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:30.200689 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-fe47ad37-ed33-425f-bcc8-da017a1933d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:30.242568 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 116/237] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:30.279972 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f28472-21c2-4748-b8b0-e689a10335b1 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:30.317447 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 122/238] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:30.352919 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe47ad37-ed33-425f-bcc8-da017a1933d5 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:30.424686 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:30.448333 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f28472-21c2-4748-b8b0-e689a10335b1 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:30.510812 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:30.518351 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb30a4a7-dd51-4ecb-99b2-afa6ac3e977d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:30.520330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30a4a7-dd51-4ecb-99b2-afa6ac3e977d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:30.520852 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30a4a7-dd51-4ecb-99b2-afa6ac3e977d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:30.521379 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30a4a7-dd51-4ecb-99b2-afa6ac3e977d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:30.536917 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe47ad37-ed33-425f-bcc8-da017a1933d5 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:30.548608 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb30a4a7-dd51-4ecb-99b2-afa6ac3e977d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:30.556166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-bb30a4a7-dd51-4ecb-99b2-afa6ac3e977d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 13:55:30.556166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:30.556166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:30.556166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:30.556166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:30.556166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:30.556166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:30.556166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:30.556166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:30.556166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:30.556166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:30.556166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:30.556166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:30.556166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:30.556166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:30.556166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:30.557096 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:30.557096 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:30.557096 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:30.557096 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:30.557096 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:30.557096 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:30.557096 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:30.557096 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:30.557096 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:30.557096 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:30.557096 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:30.557096 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:30.557096 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:30.557096 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:30.557096 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:30.557096 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:30.557913 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:30.557913 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:30.557913 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:30.557913 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:30.557913 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:30.557913 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:30.557913 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:30.557913 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:30.557913 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:30.557913 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:30.557913 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 13:55:30.557913 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:55:30.563492 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 117/239] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:55:30.567913 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb30a4a7-dd51-4ecb-99b2-afa6ac3e977d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:30.609239 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b73dd1-45d7-4e55-b885-53ff45721291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:30.611023 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b73dd1-45d7-4e55-b885-53ff45721291 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:30.611533 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b73dd1-45d7-4e55-b885-53ff45721291 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:30.613008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b73dd1-45d7-4e55-b885-53ff45721291 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:30.614440 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b73dd1-45d7-4e55-b885-53ff45721291 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:30.616254 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b73dd1-45d7-4e55-b885-53ff45721291 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:30.621581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 123/240] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:55:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:30.625557 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b73dd1-45d7-4e55-b885-53ff45721291 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:30.763204 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b73dd1-45d7-4e55-b885-53ff45721291 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:30.773190 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb30a4a7-dd51-4ecb-99b2-afa6ac3e977d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:30.802989 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccbc228-bf7c-4b5b-9287-4121a584e74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:30.803678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccbc228-bf7c-4b5b-9287-4121a584e74d None admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:30.803966 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccbc228-bf7c-4b5b-9287-4121a584e74d None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:30.804248 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccbc228-bf7c-4b5b-9287-4121a584e74d None admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:30.808935 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccbc228-bf7c-4b5b-9287-4121a584e74d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:30.808935 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccbc228-bf7c-4b5b-9287-4121a584e74d None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:30.839003 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 124/241] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 13:55:30 2022] POST /identity/v3/users => generated 293 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:55:30.849471 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44171e8-d63e-4207-a0fc-a79172a9d070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:30.850220 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44171e8-d63e-4207-a0fc-a79172a9d070 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:30.850577 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44171e8-d63e-4207-a0fc-a79172a9d070 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:30.850923 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44171e8-d63e-4207-a0fc-a79172a9d070 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:30.851749 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44171e8-d63e-4207-a0fc-a79172a9d070 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:30.852132 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44171e8-d63e-4207-a0fc-a79172a9d070 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:30.853666 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44171e8-d63e-4207-a0fc-a79172a9d070 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:30.864775 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 118/242] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 13:55:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:30.870588 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccbc228-bf7c-4b5b-9287-4121a584e74d None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:30.890621 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:31.062512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccbc228-bf7c-4b5b-9287-4121a584e74d None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:31.108883 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eaf580-f0bc-4a72-9d41-3eef481bff17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:31.110308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eaf580-f0bc-4a72-9d41-3eef481bff17 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:31.110788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eaf580-f0bc-4a72-9d41-3eef481bff17 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:31.111254 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eaf580-f0bc-4a72-9d41-3eef481bff17 None admin] PATH_INFO: `/v3/users/2432796f70bc43e7917936e387d80a6a` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:31.133828 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eaf580-f0bc-4a72-9d41-3eef481bff17 None admin] RBAC: Authorizing `identity:get_user(user_id=2432796f70bc43e7917936e387d80a6a)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:31.135584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eaf580-f0bc-4a72-9d41-3eef481bff17 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:31.137916 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 125/243] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:30 2022] GET /identity/v3/users/2432796f70bc43e7917936e387d80a6a => generated 297 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:31.141606 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f52e1-dbe5-4d7c-bab0-b79f4f217843 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:31.141890 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f52e1-dbe5-4d7c-bab0-b79f4f217843 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:31.142266 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f52e1-dbe5-4d7c-bab0-b79f4f217843 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:31.143316 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eaf580-f0bc-4a72-9d41-3eef481bff17 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:31.143737 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 119/244] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:31.149004 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85497bbc-4fe1-48a2-a4a3-9a83d3c33bef None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:31.149302 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85497bbc-4fe1-48a2-a4a3-9a83d3c33bef None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:31.149586 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85497bbc-4fe1-48a2-a4a3-9a83d3c33bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:31.172198 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-85497bbc-4fe1-48a2-a4a3-9a83d3c33bef None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:31.173458 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-85497bbc-4fe1-48a2-a4a3-9a83d3c33bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:31.185526 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:31.222361 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:31.252381 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 120/245] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:31.266704 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eaf580-f0bc-4a72-9d41-3eef481bff17 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:31.275858 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85497bbc-4fe1-48a2-a4a3-9a83d3c33bef None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:31.303648 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad80ffc-dd8d-4183-b145-01050774e9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:31.304380 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad80ffc-dd8d-4183-b145-01050774e9de None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:31.304693 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad80ffc-dd8d-4183-b145-01050774e9de None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:31.304968 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad80ffc-dd8d-4183-b145-01050774e9de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:31.380092 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad80ffc-dd8d-4183-b145-01050774e9de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:31.381708 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad80ffc-dd8d-4183-b145-01050774e9de None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:31.389047 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85497bbc-4fe1-48a2-a4a3-9a83d3c33bef None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:31.426981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041856d-5525-412d-a321-c0f05ee0c2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:31.427750 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041856d-5525-412d-a321-c0f05ee0c2e5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:31.428048 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041856d-5525-412d-a321-c0f05ee0c2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:31.428327 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041856d-5525-412d-a321-c0f05ee0c2e5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:31.439724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a041856d-5525-412d-a321-c0f05ee0c2e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:31.445314 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-a041856d-5525-412d-a321-c0f05ee0c2e5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:55:31.445314 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:31.445314 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:31.445314 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:31.445314 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:31.445314 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:31.445314 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:31.445314 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:31.445314 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:31.445314 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:31.445314 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:31.445314 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:31.445314 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:31.445314 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:31.445314 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:31.445314 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:31.446194 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:31.446194 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:31.446194 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:31.446194 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:31.446194 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:31.446194 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:31.446194 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:31.446194 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:31.446194 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:31.446194 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:31.446194 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:31.446194 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:31.446194 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:31.446194 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:31.446194 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:31.446194 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:31.448364 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:31.448364 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:31.448364 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:31.448364 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:31.448364 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:31.448364 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:31.448364 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:31.448364 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:31.448364 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:31.448364 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:31.448364 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:55:31.448364 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:55:31.448364 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 121/246] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:55:31.452758 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:31.454114 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041856d-5525-412d-a321-c0f05ee0c2e5 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:31.459267 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:31.490795 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 126/247] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:55:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:55:31.496638 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad80ffc-dd8d-4183-b145-01050774e9de None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:31.570739 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041856d-5525-412d-a321-c0f05ee0c2e5 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:31.608320 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad80ffc-dd8d-4183-b145-01050774e9de None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:31.612202 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9deb14-4c4c-42e3-9c17-94674a065ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:31.612202 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9deb14-4c4c-42e3-9c17-94674a065ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:31.612202 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9deb14-4c4c-42e3-9c17-94674a065ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:31.612543 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9deb14-4c4c-42e3-9c17-94674a065ed6 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:31.613199 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9deb14-4c4c-42e3-9c17-94674a065ed6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:31.613509 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9deb14-4c4c-42e3-9c17-94674a065ed6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:31.614898 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9deb14-4c4c-42e3-9c17-94674a065ed6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:31.626357 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 122/248] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 13:55:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:31.631981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9deb14-4c4c-42e3-9c17-94674a065ed6 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:31.647619 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc03601-1700-4825-a0bc-5180d6a50e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:31.648565 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc03601-1700-4825-a0bc-5180d6a50e9b None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:31.648983 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc03601-1700-4825-a0bc-5180d6a50e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:31.649418 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc03601-1700-4825-a0bc-5180d6a50e9b None admin] PATH_INFO: `/v3/projects/05e772d8318b47469008ecb063004d71` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:31.650783 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc03601-1700-4825-a0bc-5180d6a50e9b None admin] RBAC: Authorizing `identity:get_project(project_id=05e772d8318b47469008ecb063004d71)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:31.652971 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc03601-1700-4825-a0bc-5180d6a50e9b None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:31.655163 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 127/249] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:31 2022] GET /identity/v3/projects/05e772d8318b47469008ecb063004d71 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:31.660985 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc03601-1700-4825-a0bc-5180d6a50e9b None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:31.699805 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:31.706124 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:31.745246 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9deb14-4c4c-42e3-9c17-94674a065ed6 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:31.771616 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc03601-1700-4825-a0bc-5180d6a50e9b None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:31.782705 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7633ccd0-e5ed-445c-abc6-308693f03fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:31.783559 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7633ccd0-e5ed-445c-abc6-308693f03fcf None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:31.783786 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7633ccd0-e5ed-445c-abc6-308693f03fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:31.784074 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7633ccd0-e5ed-445c-abc6-308693f03fcf None admin] PATH_INFO: `/v3/users/f9c077f97eba4ff98d11f234d5ecc446` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:31.785300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7633ccd0-e5ed-445c-abc6-308693f03fcf None admin] RBAC: Authorizing `identity:get_user(user_id=f9c077f97eba4ff98d11f234d5ecc446)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:31.786752 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7633ccd0-e5ed-445c-abc6-308693f03fcf None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:31.788903 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 123/250] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:31 2022] GET /identity/v3/users/f9c077f97eba4ff98d11f234d5ecc446 => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:31.794259 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7633ccd0-e5ed-445c-abc6-308693f03fcf None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:31.813854 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64dc2d1-aef9-4e52-bcdc-d3c9bb818ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:31.814854 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64dc2d1-aef9-4e52-bcdc-d3c9bb818ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:31.815333 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64dc2d1-aef9-4e52-bcdc-d3c9bb818ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:31.815801 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64dc2d1-aef9-4e52-bcdc-d3c9bb818ba4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:31.820965 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64dc2d1-aef9-4e52-bcdc-d3c9bb818ba4 None admin] RBAC: Adding query filter params (role.id=6617379dc72a414bbf45eae46f9d74b2, scope.project.id=05e772d8318b47469008ecb063004d71, user.id=2432796f70bc43e7917936e387d80a6a) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:31.828526 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64dc2d1-aef9-4e52-bcdc-d3c9bb818ba4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:31.828526 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64dc2d1-aef9-4e52-bcdc-d3c9bb818ba4 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:31.830249 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 128/251] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:55:31 2022] GET /identity/v3/role_assignments?user.id=2432796f70bc43e7917936e387d80a6a&scope.project.id=05e772d8318b47469008ecb063004d71&role.id=6617379dc72a414bbf45eae46f9d74b2 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:31.903956 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7633ccd0-e5ed-445c-abc6-308693f03fcf None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:31.941274 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf04e31-033c-4f4c-ba74-df062037759c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:31.941947 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf04e31-033c-4f4c-ba74-df062037759c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:31.942233 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf04e31-033c-4f4c-ba74-df062037759c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:31.942515 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf04e31-033c-4f4c-ba74-df062037759c None admin] PATH_INFO: `/v3/projects/05e772d8318b47469008ecb063004d71` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:31.943831 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf04e31-033c-4f4c-ba74-df062037759c None admin] RBAC: Authorizing `identity:get_project(project_id=05e772d8318b47469008ecb063004d71)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:31.945434 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf04e31-033c-4f4c-ba74-df062037759c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:31.947555 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 124/252] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:31 2022] GET /identity/v3/projects/05e772d8318b47469008ecb063004d71 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:31.953953 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64dc2d1-aef9-4e52-bcdc-d3c9bb818ba4 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:32.099197 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:32.103525 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64dc2d1-aef9-4e52-bcdc-d3c9bb818ba4 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:32.118301 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25253e2-6e4c-4af6-807e-bf24b298b982 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:32.118871 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25253e2-6e4c-4af6-807e-bf24b298b982 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:32.119369 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25253e2-6e4c-4af6-807e-bf24b298b982 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:32.121826 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 125/253] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:55:32.128539 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04384f6-ecb6-4850-8d26-6c2b5d76bd35 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:32.129174 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04384f6-ecb6-4850-8d26-6c2b5d76bd35 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:32.129661 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04384f6-ecb6-4850-8d26-6c2b5d76bd35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:32.144927 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:32.157193 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7c371-e43a-41b3-b0f7-065b86634c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:32.157193 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7c371-e43a-41b3-b0f7-065b86634c64 None admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:32.157193 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7c371-e43a-41b3-b0f7-065b86634c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:32.157193 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7c371-e43a-41b3-b0f7-065b86634c64 None admin] PATH_INFO: `/v3/projects/05e772d8318b47469008ecb063004d71/users/f9c077f97eba4ff98d11f234d5ecc446/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:32.164502 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d7c371-e43a-41b3-b0f7-065b86634c64 None admin] RBAC: Authorizing `identity:create_grant(project_id=05e772d8318b47469008ecb063004d71, user_id=f9c077f97eba4ff98d11f234d5ecc446, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:32.164502 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d7c371-e43a-41b3-b0f7-065b86634c64 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:32.165792 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d04384f6-ecb6-4850-8d26-6c2b5d76bd35 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:32.167233 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d04384f6-ecb6-4850-8d26-6c2b5d76bd35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:32.195606 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 129/254] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 13:55:31 2022] PUT /identity/v3/projects/05e772d8318b47469008ecb063004d71/users/f9c077f97eba4ff98d11f234d5ecc446/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:55:32.234336 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d7467-189c-4877-855a-000246a317f2 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:32.234660 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d7467-189c-4877-855a-000246a317f2 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:32.234896 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d7467-189c-4877-855a-000246a317f2 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:32.236286 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 130/255] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:32.242035 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb5253-9993-48e5-a206-dab69fdd4a96 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:32.242316 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb5253-9993-48e5-a206-dab69fdd4a96 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:32.242573 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb5253-9993-48e5-a206-dab69fdd4a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:32.254295 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 126/256] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:55:32.273759 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-e1fb5253-9993-48e5-a206-dab69fdd4a96 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:32.275156 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-e1fb5253-9993-48e5-a206-dab69fdd4a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:32.304280 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04384f6-ecb6-4850-8d26-6c2b5d76bd35 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:32.420933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 131/257] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:55:32.443796 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04384f6-ecb6-4850-8d26-6c2b5d76bd35 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:32.461728 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb5253-9993-48e5-a206-dab69fdd4a96 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:32.520884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:32.523930 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdebaa0e-4340-47a0-9925-4a1c506ef24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:32.525162 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdebaa0e-4340-47a0-9925-4a1c506ef24e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:32.528216 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdebaa0e-4340-47a0-9925-4a1c506ef24e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:32.528768 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdebaa0e-4340-47a0-9925-4a1c506ef24e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:32.550455 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdebaa0e-4340-47a0-9925-4a1c506ef24e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:32.556729 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-bdebaa0e-4340-47a0-9925-4a1c506ef24e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:55:32.556729 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:32.556729 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:32.556729 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:32.556729 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:32.556729 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:32.556729 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:32.556729 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:32.556729 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:32.556729 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:32.556729 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:32.556729 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:32.556729 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:32.556729 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:32.556729 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:32.556729 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:32.557651 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:32.557651 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:32.557651 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:32.557651 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:32.557651 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:32.557651 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:32.557651 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:32.557651 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:32.557651 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:32.557651 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:32.557651 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:32.557651 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:32.557651 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:32.557651 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:32.557651 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:32.557651 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:32.558474 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:32.558474 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:32.558474 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:32.558474 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:32.558474 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:32.558474 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:32.558474 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:32.558474 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:32.558474 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:32.558474 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:32.558474 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:55:32.558474 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:55:32.564378 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 127/258] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 13:55:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:55:32.570416 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdebaa0e-4340-47a0-9925-4a1c506ef24e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:32.639839 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:32.662389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb5253-9993-48e5-a206-dab69fdd4a96 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:32.726863 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c3d3c-b689-444f-ac1c-9831b97d1a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:32.727927 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c3d3c-b689-444f-ac1c-9831b97d1a72 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:32.728430 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c3d3c-b689-444f-ac1c-9831b97d1a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:32.728933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c3d3c-b689-444f-ac1c-9831b97d1a72 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:32.748273 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0c3d3c-b689-444f-ac1c-9831b97d1a72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:32.753698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-fa0c3d3c-b689-444f-ac1c-9831b97d1a72 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:55:32.753698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:32.753698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:32.753698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:32.753698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:32.753698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:32.753698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:32.753698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:32.753698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:32.753698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:32.753698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:32.753698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:32.753698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:32.753698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:32.753698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:32.753698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:32.754562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:32.754562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:32.754562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:32.754562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:32.754562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:32.754562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:32.754562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:32.754562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:32.754562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:32.754562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:32.754562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:32.754562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:32.754562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:32.754562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:32.754562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:32.754562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:32.755314 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:32.755314 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:32.755314 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:32.755314 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:32.755314 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:32.755314 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:32.755314 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:32.755314 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:32.755314 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:32.755314 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:32.755314 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:55:32.755314 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:55:32.757953 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 132/259] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 13:55:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:32.764501 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c3d3c-b689-444f-ac1c-9831b97d1a72 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:32.847303 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdebaa0e-4340-47a0-9925-4a1c506ef24e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:32.938100 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c838313-f826-493d-90d7-b9e5330d37b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:32.939057 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c838313-f826-493d-90d7-b9e5330d37b0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:32.939518 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c838313-f826-493d-90d7-b9e5330d37b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:32.939960 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c838313-f826-493d-90d7-b9e5330d37b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:32.940882 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c838313-f826-493d-90d7-b9e5330d37b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:32.941352 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c838313-f826-493d-90d7-b9e5330d37b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:32.942909 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c838313-f826-493d-90d7-b9e5330d37b0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:32.963003 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 128/260] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:32.969129 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c838313-f826-493d-90d7-b9e5330d37b0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:32.972333 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c3d3c-b689-444f-ac1c-9831b97d1a72 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:33.045444 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690afe91-5bd5-4fa8-a195-3b006674238a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:33.046118 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690afe91-5bd5-4fa8-a195-3b006674238a None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:33.046474 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690afe91-5bd5-4fa8-a195-3b006674238a None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:33.046935 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690afe91-5bd5-4fa8-a195-3b006674238a None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:33.047746 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690afe91-5bd5-4fa8-a195-3b006674238a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:33.048124 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690afe91-5bd5-4fa8-a195-3b006674238a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:33.049643 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690afe91-5bd5-4fa8-a195-3b006674238a None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:33.067614 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 133/261] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:55:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:33.073417 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690afe91-5bd5-4fa8-a195-3b006674238a None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:33.105883 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:33.106174 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:33.226592 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c838313-f826-493d-90d7-b9e5330d37b0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:33.266254 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690afe91-5bd5-4fa8-a195-3b006674238a None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:33.318585 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b225a1a3-eb67-4092-ad3b-ca2be1f21d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:33.319167 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225a1a3-eb67-4092-ad3b-ca2be1f21d59 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:33.319450 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225a1a3-eb67-4092-ad3b-ca2be1f21d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:33.319726 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225a1a3-eb67-4092-ad3b-ca2be1f21d59 None admin] PATH_INFO: `/v3/users/a61d02fa4d1c4567a197ea1027e1ffdc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:33.325188 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b225a1a3-eb67-4092-ad3b-ca2be1f21d59 None admin] RBAC: Authorizing `identity:get_user(user_id=a61d02fa4d1c4567a197ea1027e1ffdc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:33.326638 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b225a1a3-eb67-4092-ad3b-ca2be1f21d59 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:33.328591 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 129/262] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:32 2022] GET /identity/v3/users/a61d02fa4d1c4567a197ea1027e1ffdc => generated 283 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:33.333820 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b225a1a3-eb67-4092-ad3b-ca2be1f21d59 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:33.334176 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e9c58-1dea-4617-82bf-a3381b959dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:33.334837 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e9c58-1dea-4617-82bf-a3381b959dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:33.335126 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e9c58-1dea-4617-82bf-a3381b959dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:33.335410 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e9c58-1dea-4617-82bf-a3381b959dd1 None admin] PATH_INFO: `/v3/users/a61d02fa4d1c4567a197ea1027e1ffdc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:33.340781 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012e9c58-1dea-4617-82bf-a3381b959dd1 None admin] RBAC: Authorizing `identity:get_user(user_id=a61d02fa4d1c4567a197ea1027e1ffdc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:33.342308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012e9c58-1dea-4617-82bf-a3381b959dd1 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:33.344057 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 134/263] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:33 2022] GET /identity/v3/users/a61d02fa4d1c4567a197ea1027e1ffdc => generated 283 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:33.350147 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e9c58-1dea-4617-82bf-a3381b959dd1 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:33.504219 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:33.556264 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e9c58-1dea-4617-82bf-a3381b959dd1 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:33.575326 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b225a1a3-eb67-4092-ad3b-ca2be1f21d59 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:33.627592 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:33.656511 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ce99c-51f9-4725-b59f-826687e3bd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:33.657174 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ce99c-51f9-4725-b59f-826687e3bd3d None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:33.657465 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ce99c-51f9-4725-b59f-826687e3bd3d None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:33.657745 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ce99c-51f9-4725-b59f-826687e3bd3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:33.669393 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8294b1-4fb6-4c6f-b82a-f50576006429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:33.670062 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8294b1-4fb6-4c6f-b82a-f50576006429 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:33.670352 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8294b1-4fb6-4c6f-b82a-f50576006429 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:33.670637 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8294b1-4fb6-4c6f-b82a-f50576006429 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:33.761402 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8294b1-4fb6-4c6f-b82a-f50576006429 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:33.762885 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8294b1-4fb6-4c6f-b82a-f50576006429 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:33.838618 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1ce99c-51f9-4725-b59f-826687e3bd3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:33.840183 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1ce99c-51f9-4725-b59f-826687e3bd3d None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:33.869210 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 130/264] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:55:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 13:55:33.875329 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8294b1-4fb6-4c6f-b82a-f50576006429 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:33.899843 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:33.924908 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:33.943832 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 135/265] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:55:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:55:33.949424 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ce99c-51f9-4725-b59f-826687e3bd3d None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:33.982392 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8294b1-4fb6-4c6f-b82a-f50576006429 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:34.041566 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08461807-cd89-4caa-930d-f9529287b7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:34.043057 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08461807-cd89-4caa-930d-f9529287b7f8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:34.043492 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08461807-cd89-4caa-930d-f9529287b7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:34.043925 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08461807-cd89-4caa-930d-f9529287b7f8 None admin] PATH_INFO: `/v3/projects/05e772d8318b47469008ecb063004d71` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:34.052505 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08461807-cd89-4caa-930d-f9529287b7f8 None admin] RBAC: Authorizing `identity:get_project(project_id=05e772d8318b47469008ecb063004d71)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:34.056363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08461807-cd89-4caa-930d-f9529287b7f8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:34.058521 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 131/266] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:33 2022] GET /identity/v3/projects/05e772d8318b47469008ecb063004d71 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:34.073759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08461807-cd89-4caa-930d-f9529287b7f8 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:34.127195 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ce99c-51f9-4725-b59f-826687e3bd3d None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:34.203838 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87591b50-1dd8-4f5a-b0c1-d3d4bfb3d97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:34.204514 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87591b50-1dd8-4f5a-b0c1-d3d4bfb3d97a None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:34.204804 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87591b50-1dd8-4f5a-b0c1-d3d4bfb3d97a None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:34.205080 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87591b50-1dd8-4f5a-b0c1-d3d4bfb3d97a None admin] PATH_INFO: `/v3/projects/05e772d8318b47469008ecb063004d71` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:34.210458 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87591b50-1dd8-4f5a-b0c1-d3d4bfb3d97a None admin] RBAC: Authorizing `identity:get_project(project_id=05e772d8318b47469008ecb063004d71)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:34.211945 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87591b50-1dd8-4f5a-b0c1-d3d4bfb3d97a None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:34.220259 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 136/267] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:33 2022] GET /identity/v3/projects/05e772d8318b47469008ecb063004d71 => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:34.242036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:34.249000 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87591b50-1dd8-4f5a-b0c1-d3d4bfb3d97a None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:34.269872 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08461807-cd89-4caa-930d-f9529287b7f8 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:34.342865 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb06dd-197b-4251-86be-6f35e56837aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:34.344669 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb06dd-197b-4251-86be-6f35e56837aa None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:34.344956 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb06dd-197b-4251-86be-6f35e56837aa None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:34.345235 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb06dd-197b-4251-86be-6f35e56837aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:34.346042 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb06dd-197b-4251-86be-6f35e56837aa None admin] RBAC: Adding query filter params (role.id=58f91e3a13664dabb18c6227a1658fa6, scope.project.id=05e772d8318b47469008ecb063004d71, user.id=a61d02fa4d1c4567a197ea1027e1ffdc) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:34.346355 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb06dd-197b-4251-86be-6f35e56837aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:34.347799 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb06dd-197b-4251-86be-6f35e56837aa None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:34.356854 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:34.365623 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 132/268] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:55:34 2022] GET /identity/v3/role_assignments?user.id=a61d02fa4d1c4567a197ea1027e1ffdc&scope.project.id=05e772d8318b47469008ecb063004d71&role.id=58f91e3a13664dabb18c6227a1658fa6 => generated 608 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:34.366922 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13fe06-3af0-41a9-8e1e-993ea88c6df0 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:34.367223 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13fe06-3af0-41a9-8e1e-993ea88c6df0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:34.367490 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13fe06-3af0-41a9-8e1e-993ea88c6df0 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:34.368932 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 133/269] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:34.378155 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60e2c0-1816-45df-b334-807d9d0dba9f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:34.378432 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60e2c0-1816-45df-b334-807d9d0dba9f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:34.378636 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60e2c0-1816-45df-b334-807d9d0dba9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:34.408556 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8c60e2c0-1816-45df-b334-807d9d0dba9f None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:34.409799 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8c60e2c0-1816-45df-b334-807d9d0dba9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:34.421024 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87591b50-1dd8-4f5a-b0c1-d3d4bfb3d97a None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:34.468150 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb219875-6b9e-4f94-979b-aaf37ffb7d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:34.468819 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb219875-6b9e-4f94-979b-aaf37ffb7d42 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:34.469095 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb219875-6b9e-4f94-979b-aaf37ffb7d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:34.469369 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb219875-6b9e-4f94-979b-aaf37ffb7d42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:34.470176 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb219875-6b9e-4f94-979b-aaf37ffb7d42 None admin] RBAC: Adding query filter params (role.id=47e170213d8049a28f8b8a0d736fdf6b, scope.project.id=05e772d8318b47469008ecb063004d71, user.id=a61d02fa4d1c4567a197ea1027e1ffdc) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:34.470487 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb219875-6b9e-4f94-979b-aaf37ffb7d42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:34.471927 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb219875-6b9e-4f94-979b-aaf37ffb7d42 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:34.487096 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 137/270] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:55:34 2022] GET /identity/v3/role_assignments?user.id=a61d02fa4d1c4567a197ea1027e1ffdc&scope.project.id=05e772d8318b47469008ecb063004d71&role.id=47e170213d8049a28f8b8a0d736fdf6b => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:34.554348 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 134/271] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:34.591628 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb219875-6b9e-4f94-979b-aaf37ffb7d42 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:34.620746 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cbfd50-e9ca-4b66-bca5-6b49e7ce19ba None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:34.621257 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cbfd50-e9ca-4b66-bca5-6b49e7ce19ba None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:34.621332 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cbfd50-e9ca-4b66-bca5-6b49e7ce19ba None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:34.623698 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 135/272] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:34.629226 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3dce0-39d3-4f91-87cb-6bac9a29f5d6 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:34.629554 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3dce0-39d3-4f91-87cb-6bac9a29f5d6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:34.629717 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3dce0-39d3-4f91-87cb-6bac9a29f5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:34.664939 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-83e3dce0-39d3-4f91-87cb-6bac9a29f5d6 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:34.666169 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-83e3dce0-39d3-4f91-87cb-6bac9a29f5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:34.777995 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 136/273] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:55:34.780884 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb219875-6b9e-4f94-979b-aaf37ffb7d42 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:34.791657 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:34.808503 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e3dce0-39d3-4f91-87cb-6bac9a29f5d6 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:34.820443 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142211ad-dce6-4980-a1c0-a748b4ae614c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:34.821139 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142211ad-dce6-4980-a1c0-a748b4ae614c None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:34.821420 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142211ad-dce6-4980-a1c0-a748b4ae614c None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:34.821696 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142211ad-dce6-4980-a1c0-a748b4ae614c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:34.822852 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142211ad-dce6-4980-a1c0-a748b4ae614c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:34.827762 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:34.828937 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142211ad-dce6-4980-a1c0-a748b4ae614c None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:34.831132 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 138/274] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:55:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:34.836561 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142211ad-dce6-4980-a1c0-a748b4ae614c None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:34.934081 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e3dce0-39d3-4f91-87cb-6bac9a29f5d6 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:34.972158 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142211ad-dce6-4980-a1c0-a748b4ae614c None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:34.978103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231173cb-37f2-4a8f-95e9-48f7a1b7e72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:34.978756 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231173cb-37f2-4a8f-95e9-48f7a1b7e72e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:34.979035 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231173cb-37f2-4a8f-95e9-48f7a1b7e72e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:34.979307 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231173cb-37f2-4a8f-95e9-48f7a1b7e72e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:34.989669 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231173cb-37f2-4a8f-95e9-48f7a1b7e72e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:34.994518 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-231173cb-37f2-4a8f-95e9-48f7a1b7e72e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:55:34.994518 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:34.994518 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:34.994518 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:34.994518 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:34.994518 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:34.994518 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:34.994518 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:34.994518 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:34.994518 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:34.994518 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:34.994518 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:34.994518 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:34.994518 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:34.994518 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:34.994518 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:34.995372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:34.995372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:34.995372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:34.995372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:34.995372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:34.995372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:34.995372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:34.995372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:34.995372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:34.995372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:34.995372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:34.995372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:34.995372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:34.995372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:34.995372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:34.995372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:34.996699 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:34.996699 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:34.996699 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:34.996699 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:34.996699 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:34.996699 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:34.996699 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:34.996699 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:34.996699 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:34.996699 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:34.996699 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:55:34.996699 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:55:34.996699 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 137/275] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:34.997959 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848076a4-dca8-4d30-8f5e-83c32115e82e None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:34.997959 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848076a4-dca8-4d30-8f5e-83c32115e82e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:34.997959 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848076a4-dca8-4d30-8f5e-83c32115e82e None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:34.999360 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 138/276] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:35.001694 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848076a4-dca8-4d30-8f5e-83c32115e82e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:35.018668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb8de23-35ef-4d33-9d3b-fb881f3f8beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:35.020087 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8de23-35ef-4d33-9d3b-fb881f3f8beb None admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:35.020898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8de23-35ef-4d33-9d3b-fb881f3f8beb None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:35.021383 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8de23-35ef-4d33-9d3b-fb881f3f8beb None admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:35.022528 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb8de23-35ef-4d33-9d3b-fb881f3f8beb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:35.024669 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb8de23-35ef-4d33-9d3b-fb881f3f8beb None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:35.049246 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 139/277] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 13:55:34 2022] POST /identity/v3/users => generated 293 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:55:35.049246 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4276e2a5-b232-4cb9-8670-66c23f485faa None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:35.049246 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4276e2a5-b232-4cb9-8670-66c23f485faa None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:35.049246 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4276e2a5-b232-4cb9-8670-66c23f485faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:35.086799 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-4276e2a5-b232-4cb9-8670-66c23f485faa None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:35.088203 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-4276e2a5-b232-4cb9-8670-66c23f485faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:35.100672 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:35.128287 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:35.134907 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848076a4-dca8-4d30-8f5e-83c32115e82e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:35.172876 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4038749b-5053-49c0-abee-9decec3aa8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:35.173529 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038749b-5053-49c0-abee-9decec3aa8d2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:35.173816 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038749b-5053-49c0-abee-9decec3aa8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:35.174096 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038749b-5053-49c0-abee-9decec3aa8d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:35.174838 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4038749b-5053-49c0-abee-9decec3aa8d2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:35.175151 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4038749b-5053-49c0-abee-9decec3aa8d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:35.176614 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4038749b-5053-49c0-abee-9decec3aa8d2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:35.180828 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 140/278] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:35.181962 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53abe3ae-b9c8-41e9-8aec-0c952987729a None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:35.182250 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53abe3ae-b9c8-41e9-8aec-0c952987729a None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:35.182885 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53abe3ae-b9c8-41e9-8aec-0c952987729a None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:35.184576 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 141/279] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:35.189875 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54e8c8-d468-4061-bb3b-5a309244e95a None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:35.190173 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54e8c8-d468-4061-bb3b-5a309244e95a None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:35.190461 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54e8c8-d468-4061-bb3b-5a309244e95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:35.195303 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 139/280] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 13:55:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:35.202500 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4038749b-5053-49c0-abee-9decec3aa8d2 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:35.214364 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-fe54e8c8-d468-4061-bb3b-5a309244e95a None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:35.215570 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-fe54e8c8-d468-4061-bb3b-5a309244e95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:35.298640 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 142/281] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:35.300188 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54e8c8-d468-4061-bb3b-5a309244e95a None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:35.320325 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4038749b-5053-49c0-abee-9decec3aa8d2 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:35.360385 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a72fa15-c226-4a9d-bc88-3f10e29e16e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:35.361242 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72fa15-c226-4a9d-bc88-3f10e29e16e0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:35.361550 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72fa15-c226-4a9d-bc88-3f10e29e16e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:35.361807 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72fa15-c226-4a9d-bc88-3f10e29e16e0 None admin] PATH_INFO: `/v3/users/7f6b3483a62f4e4587c64b29663b856a` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:35.365616 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:35.380808 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a72fa15-c226-4a9d-bc88-3f10e29e16e0 None admin] RBAC: Authorizing `identity:get_user(user_id=7f6b3483a62f4e4587c64b29663b856a)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:35.382272 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a72fa15-c226-4a9d-bc88-3f10e29e16e0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:35.384532 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 140/282] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:35 2022] GET /identity/v3/users/7f6b3483a62f4e4587c64b29663b856a => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:35.385333 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a72fa15-c226-4a9d-bc88-3f10e29e16e0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:35.423278 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:35.431603 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54e8c8-d468-4061-bb3b-5a309244e95a None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:35.468661 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105dbaee-878a-461a-b098-1ee11c8090e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:35.469373 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105dbaee-878a-461a-b098-1ee11c8090e4 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:35.469738 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105dbaee-878a-461a-b098-1ee11c8090e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:35.470086 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105dbaee-878a-461a-b098-1ee11c8090e4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:35.482052 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105dbaee-878a-461a-b098-1ee11c8090e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:35.487258 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-105dbaee-878a-461a-b098-1ee11c8090e4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 13:55:35.487258 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:35.487258 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:35.487258 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:35.487258 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:35.487258 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:35.487258 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:35.487258 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:35.487258 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:35.487258 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:35.487258 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:35.487258 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:35.487258 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:35.487258 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:35.487258 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:35.487258 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:35.488144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:35.488144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:35.488144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:35.488144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:35.488144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:35.488144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:35.488144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:35.488144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:35.488144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:35.488144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:35.488144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:35.488144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:35.488144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:35.488144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:35.488144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:35.488144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:35.488954 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:35.488954 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:35.488954 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:35.488954 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:35.488954 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:35.488954 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:35.488954 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:35.488954 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:35.488954 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:35.488954 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:35.488954 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 13:55:35.488954 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:55:35.488954 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 143/283] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:35.489571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105dbaee-878a-461a-b098-1ee11c8090e4 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:35.524859 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a72fa15-c226-4a9d-bc88-3f10e29e16e0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:35.562376 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb6e8c0-b3aa-45ef-b8a4-b92c5a80ad3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:35.563173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6e8c0-b3aa-45ef-b8a4-b92c5a80ad3d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:35.563567 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6e8c0-b3aa-45ef-b8a4-b92c5a80ad3d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:35.563944 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6e8c0-b3aa-45ef-b8a4-b92c5a80ad3d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:35.576265 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb6e8c0-b3aa-45ef-b8a4-b92c5a80ad3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:35.581584 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-deb6e8c0-b3aa-45ef-b8a4-b92c5a80ad3d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:55:35.581584 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:35.581584 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:35.581584 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:35.581584 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:35.581584 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:35.581584 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:35.581584 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:35.581584 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:35.581584 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:35.581584 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:35.581584 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:35.581584 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:35.581584 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:35.581584 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:35.581584 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:35.582642 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:35.582642 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:35.582642 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:35.582642 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:35.582642 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:35.582642 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:35.582642 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:35.582642 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:35.582642 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:35.582642 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:35.582642 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:35.582642 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:35.582642 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:35.582642 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:35.582642 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:35.582642 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:35.583600 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:35.583600 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:35.583600 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:35.583600 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:35.583600 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:35.583600 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:35.583600 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:35.583600 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:35.583600 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:35.583600 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:35.583600 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:55:35.583600 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:55:35.585537 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 141/284] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:35.591535 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb6e8c0-b3aa-45ef-b8a4-b92c5a80ad3d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:35.630522 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105dbaee-878a-461a-b098-1ee11c8090e4 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:35.664554 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:35.671708 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39990a81-daf0-4b13-9f12-ff08a6094ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:35.673119 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39990a81-daf0-4b13-9f12-ff08a6094ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:35.673931 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39990a81-daf0-4b13-9f12-ff08a6094ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:35.674449 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39990a81-daf0-4b13-9f12-ff08a6094ee6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:35.675714 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39990a81-daf0-4b13-9f12-ff08a6094ee6 None admin] RBAC: Adding query filter params (role.id=926b9f36c24847c582c84af904250abc, user.id=7f6b3483a62f4e4587c64b29663b856a, scope.system=all) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:35.676358 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39990a81-daf0-4b13-9f12-ff08a6094ee6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:35.679060 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39990a81-daf0-4b13-9f12-ff08a6094ee6 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:35.684857 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 144/285] 10.222.0.48 () {56 vars in 1308 bytes} [Mon Aug 1 13:55:35 2022] GET /identity/v3/role_assignments?user.id=7f6b3483a62f4e4587c64b29663b856a&scope.system=all&role.id=926b9f36c24847c582c84af904250abc => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:35.685800 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39990a81-daf0-4b13-9f12-ff08a6094ee6 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:35.707618 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:35.774821 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb6e8c0-b3aa-45ef-b8a4-b92c5a80ad3d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:35.817200 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39990a81-daf0-4b13-9f12-ff08a6094ee6 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:35.860864 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b2b12-de4f-48eb-aefe-bce253f1f3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:35.861927 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b2b12-de4f-48eb-aefe-bce253f1f3f3 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:35.862356 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b2b12-de4f-48eb-aefe-bce253f1f3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:35.862849 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b2b12-de4f-48eb-aefe-bce253f1f3f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:35.863740 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b2b12-de4f-48eb-aefe-bce253f1f3f3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:35.864196 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b2b12-de4f-48eb-aefe-bce253f1f3f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:35.865940 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b2b12-de4f-48eb-aefe-bce253f1f3f3 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:35.876906 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 145/286] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 13:55:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:35.882410 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b2b12-de4f-48eb-aefe-bce253f1f3f3 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:35.885199 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3025e17-3bd3-45ce-ad17-9b723d278f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:35.885834 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3025e17-3bd3-45ce-ad17-9b723d278f42 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:35.886796 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3025e17-3bd3-45ce-ad17-9b723d278f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:35.886939 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3025e17-3bd3-45ce-ad17-9b723d278f42 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:35.887678 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3025e17-3bd3-45ce-ad17-9b723d278f42 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:35.887987 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3025e17-3bd3-45ce-ad17-9b723d278f42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:35.889417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3025e17-3bd3-45ce-ad17-9b723d278f42 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:35.908010 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 142/287] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 13:55:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:35.913726 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3025e17-3bd3-45ce-ad17-9b723d278f42 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:35.991253 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:36.014012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b2b12-de4f-48eb-aefe-bce253f1f3f3 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:36.052642 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f36ac7-245c-4adb-8212-f91845cde0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:36.053714 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f36ac7-245c-4adb-8212-f91845cde0e1 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:36.054144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f36ac7-245c-4adb-8212-f91845cde0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:36.054585 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f36ac7-245c-4adb-8212-f91845cde0e1 None admin] PATH_INFO: `/v3/users/f9c077f97eba4ff98d11f234d5ecc446` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:36.055957 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f36ac7-245c-4adb-8212-f91845cde0e1 None admin] RBAC: Authorizing `identity:get_user(user_id=f9c077f97eba4ff98d11f234d5ecc446)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:36.057876 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f36ac7-245c-4adb-8212-f91845cde0e1 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:36.060532 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 146/288] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:35 2022] GET /identity/v3/users/f9c077f97eba4ff98d11f234d5ecc446 => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:36.065646 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f36ac7-245c-4adb-8212-f91845cde0e1 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:36.103451 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3025e17-3bd3-45ce-ad17-9b723d278f42 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:36.141703 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:36.177580 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decce74f-dd2f-454d-a167-fb0f932d4715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:36.178227 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decce74f-dd2f-454d-a167-fb0f932d4715 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:36.178508 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decce74f-dd2f-454d-a167-fb0f932d4715 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:36.178789 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decce74f-dd2f-454d-a167-fb0f932d4715 None admin] PATH_INFO: `/v3/users/2432796f70bc43e7917936e387d80a6a` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:36.180615 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decce74f-dd2f-454d-a167-fb0f932d4715 None admin] RBAC: Authorizing `identity:get_user(user_id=2432796f70bc43e7917936e387d80a6a)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:36.182062 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decce74f-dd2f-454d-a167-fb0f932d4715 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:36.185795 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 143/289] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:35 2022] GET /identity/v3/users/2432796f70bc43e7917936e387d80a6a => generated 297 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:36.192943 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decce74f-dd2f-454d-a167-fb0f932d4715 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:36.257484 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f36ac7-245c-4adb-8212-f91845cde0e1 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:36.310433 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f2a53f-3d0b-4d11-ab10-540023851c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:36.311009 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2a53f-3d0b-4d11-ab10-540023851c86 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:36.311422 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2a53f-3d0b-4d11-ab10-540023851c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:36.311708 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2a53f-3d0b-4d11-ab10-540023851c86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:36.338115 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:36.344279 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decce74f-dd2f-454d-a167-fb0f932d4715 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:36.386731 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4aca8a-f49e-46cd-abee-3cba280e5849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:36.387402 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4aca8a-f49e-46cd-abee-3cba280e5849 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:36.387708 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4aca8a-f49e-46cd-abee-3cba280e5849 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:36.387952 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4aca8a-f49e-46cd-abee-3cba280e5849 None admin] PATH_INFO: `/v3/projects/05e772d8318b47469008ecb063004d71` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:36.393294 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4aca8a-f49e-46cd-abee-3cba280e5849 None admin] RBAC: Authorizing `identity:get_project(project_id=05e772d8318b47469008ecb063004d71)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:36.394768 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4aca8a-f49e-46cd-abee-3cba280e5849 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:36.396510 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 144/290] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:36 2022] GET /identity/v3/projects/05e772d8318b47469008ecb063004d71 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:36.402771 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4aca8a-f49e-46cd-abee-3cba280e5849 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:36.405908 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f2a53f-3d0b-4d11-ab10-540023851c86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:36.407672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f2a53f-3d0b-4d11-ab10-540023851c86 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:36.519669 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 147/291] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:55:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:55:36.520720 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:36.527509 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f2a53f-3d0b-4d11-ab10-540023851c86 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:36.580834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:36.647481 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4aca8a-f49e-46cd-abee-3cba280e5849 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:36.653442 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f2a53f-3d0b-4d11-ab10-540023851c86 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:36.694732 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c024e-be4d-49f8-804a-75a4629614f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:36.695790 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c024e-be4d-49f8-804a-75a4629614f4 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:36.696265 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c024e-be4d-49f8-804a-75a4629614f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:36.697632 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c024e-be4d-49f8-804a-75a4629614f4 None admin] PATH_INFO: `/v3/projects/05e772d8318b47469008ecb063004d71` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:36.699186 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c024e-be4d-49f8-804a-75a4629614f4 None admin] RBAC: Authorizing `identity:get_project(project_id=05e772d8318b47469008ecb063004d71)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:36.701980 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c024e-be4d-49f8-804a-75a4629614f4 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:36.704965 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 148/292] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:36 2022] GET /identity/v3/projects/05e772d8318b47469008ecb063004d71 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:36.713442 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c024e-be4d-49f8-804a-75a4629614f4 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:36.732840 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9038b0-2a4b-4c8e-bf79-e246e018075a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:36.732840 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9038b0-2a4b-4c8e-bf79-e246e018075a None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:36.732840 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9038b0-2a4b-4c8e-bf79-e246e018075a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:36.732840 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9038b0-2a4b-4c8e-bf79-e246e018075a None admin] PATH_INFO: `/v3/projects/05e772d8318b47469008ecb063004d71/users/2432796f70bc43e7917936e387d80a6a/roles/6617379dc72a414bbf45eae46f9d74b2` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:36.733527 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9038b0-2a4b-4c8e-bf79-e246e018075a None admin] RBAC: Authorizing `identity:create_grant(project_id=05e772d8318b47469008ecb063004d71, user_id=2432796f70bc43e7917936e387d80a6a, role_id=6617379dc72a414bbf45eae46f9d74b2)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:36.733527 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9038b0-2a4b-4c8e-bf79-e246e018075a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:36.764024 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 145/293] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 13:55:36 2022] PUT /identity/v3/projects/05e772d8318b47469008ecb063004d71/users/2432796f70bc43e7917936e387d80a6a/roles/6617379dc72a414bbf45eae46f9d74b2 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 13:55:36.837639 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c024e-be4d-49f8-804a-75a4629614f4 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:36.870986 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:36.896949 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41fee9f-1be2-49f2-9793-9a7ff36d52db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:36.897589 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41fee9f-1be2-49f2-9793-9a7ff36d52db None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:36.897877 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41fee9f-1be2-49f2-9793-9a7ff36d52db None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:36.898156 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41fee9f-1be2-49f2-9793-9a7ff36d52db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:36.898976 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41fee9f-1be2-49f2-9793-9a7ff36d52db None admin] RBAC: Adding query filter params (role.id=926b9f36c24847c582c84af904250abc, scope.project.id=05e772d8318b47469008ecb063004d71, user.id=f9c077f97eba4ff98d11f234d5ecc446) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:36.899291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41fee9f-1be2-49f2-9793-9a7ff36d52db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:36.900766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41fee9f-1be2-49f2-9793-9a7ff36d52db None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:36.907937 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 149/294] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:55:36 2022] GET /identity/v3/role_assignments?user.id=f9c077f97eba4ff98d11f234d5ecc446&scope.project.id=05e772d8318b47469008ecb063004d71&role.id=926b9f36c24847c582c84af904250abc => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:37.474119 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9bd492-a0a0-487c-a8d1-abc46610a05e None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:37.474448 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9bd492-a0a0-487c-a8d1-abc46610a05e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:37.474691 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9bd492-a0a0-487c-a8d1-abc46610a05e None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:37.476290 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 146/295] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:37.481879 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034444d-b07a-40ca-b4be-10c28c373fd1 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:37.482171 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034444d-b07a-40ca-b4be-10c28c373fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:37.482450 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034444d-b07a-40ca-b4be-10c28c373fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:37.504816 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-d034444d-b07a-40ca-b4be-10c28c373fd1 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:37.506088 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-d034444d-b07a-40ca-b4be-10c28c373fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:37.549304 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2815c-93eb-4b9b-b777-7ea2aa91d146 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:37.549914 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2815c-93eb-4b9b-b777-7ea2aa91d146 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:37.550389 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2815c-93eb-4b9b-b777-7ea2aa91d146 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:37.551920 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 147/296] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:55:37.557435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204d60c-d4d5-4966-acc3-2e8fa0c8c954 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:37.558007 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204d60c-d4d5-4966-acc3-2e8fa0c8c954 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:37.558496 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204d60c-d4d5-4966-acc3-2e8fa0c8c954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:37.579760 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b204d60c-d4d5-4966-acc3-2e8fa0c8c954 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:37.581822 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b204d60c-d4d5-4966-acc3-2e8fa0c8c954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:37.590311 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 150/297] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:55:37.613760 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034444d-b07a-40ca-b4be-10c28c373fd1 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:37.662126 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 148/298] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:37.686538 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204d60c-d4d5-4966-acc3-2e8fa0c8c954 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:37.735019 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:37.744433 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034444d-b07a-40ca-b4be-10c28c373fd1 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:37.761781 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:37.803824 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204d60c-d4d5-4966-acc3-2e8fa0c8c954 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:37.807868 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402921bb-342a-4585-b45e-9038cb413e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:37.808785 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402921bb-342a-4585-b45e-9038cb413e7c None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:37.809217 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402921bb-342a-4585-b45e-9038cb413e7c None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:37.809682 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402921bb-342a-4585-b45e-9038cb413e7c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:37.826938 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402921bb-342a-4585-b45e-9038cb413e7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:37.833990 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-402921bb-342a-4585-b45e-9038cb413e7c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 13:55:37.833990 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:37.833990 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:37.833990 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:37.833990 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:37.833990 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:37.833990 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:37.833990 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:37.833990 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:37.833990 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:37.833990 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:37.833990 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:37.833990 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:37.833990 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:37.833990 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:37.833990 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:37.835034 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:37.835034 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:37.835034 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:37.835034 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:37.835034 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:37.835034 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:37.835034 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:37.835034 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:37.835034 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:37.835034 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:37.835034 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:37.835034 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:37.835034 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:37.835034 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:37.835034 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:37.835034 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:37.835977 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:37.835977 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:37.835977 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:37.835977 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:37.835977 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:37.835977 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:37.835977 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:37.835977 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:37.835977 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:37.835977 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:37.835977 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 13:55:37.835977 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:55:37.842717 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 151/299] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:55:37.844858 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402921bb-342a-4585-b45e-9038cb413e7c None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:37.846911 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c892e1-cdf5-480c-8d4a-a0fec4a8d01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:37.847740 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c892e1-cdf5-480c-8d4a-a0fec4a8d01c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:37.848156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c892e1-cdf5-480c-8d4a-a0fec4a8d01c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:37.848558 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c892e1-cdf5-480c-8d4a-a0fec4a8d01c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:37.849931 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c892e1-cdf5-480c-8d4a-a0fec4a8d01c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:37.851600 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c892e1-cdf5-480c-8d4a-a0fec4a8d01c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:37.853325 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 149/300] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:55:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:37.859552 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c892e1-cdf5-480c-8d4a-a0fec4a8d01c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:37.951541 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402921bb-342a-4585-b45e-9038cb413e7c None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:37.968039 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c892e1-cdf5-480c-8d4a-a0fec4a8d01c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:37.980576 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:37.988052 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041d880-e077-4234-9331-cfcc7ec2d6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:37.988894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041d880-e077-4234-9331-cfcc7ec2d6d7 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:37.989345 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041d880-e077-4234-9331-cfcc7ec2d6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:37.989768 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041d880-e077-4234-9331-cfcc7ec2d6d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:37.990687 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4041d880-e077-4234-9331-cfcc7ec2d6d7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:37.991175 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4041d880-e077-4234-9331-cfcc7ec2d6d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:37.992768 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4041d880-e077-4234-9331-cfcc7ec2d6d7 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:38.001914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 152/301] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 13:55:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:38.003126 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d4f8f-96e2-4de1-bcba-633b58076ad9 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:38.003420 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d4f8f-96e2-4de1-bcba-633b58076ad9 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:38.003703 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d4f8f-96e2-4de1-bcba-633b58076ad9 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:38.005764 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 153/302] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:38.010612 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d4f8f-96e2-4de1-bcba-633b58076ad9 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:38.011631 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0629bf7e-d77a-48bf-bb87-bb4dc438aae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:38.012269 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629bf7e-d77a-48bf-bb87-bb4dc438aae3 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:38.012575 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629bf7e-d77a-48bf-bb87-bb4dc438aae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:38.012868 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629bf7e-d77a-48bf-bb87-bb4dc438aae3 None admin] PATH_INFO: `/v3/groups` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:38.013533 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0629bf7e-d77a-48bf-bb87-bb4dc438aae3 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:38.014950 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0629bf7e-d77a-48bf-bb87-bb4dc438aae3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:38.031787 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 150/303] 10.222.0.48 () {60 vars in 1134 bytes} [Mon Aug 1 13:55:37 2022] POST /identity/v3/groups => generated 229 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 13:55:38.042063 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436257b-e0a2-447f-b4fa-101136f5ebe8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:38.042876 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436257b-e0a2-447f-b4fa-101136f5ebe8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:38.043155 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436257b-e0a2-447f-b4fa-101136f5ebe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:38.068379 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-7436257b-e0a2-447f-b4fa-101136f5ebe8 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:38.069809 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7436257b-e0a2-447f-b4fa-101136f5ebe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:38.070272 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:38.133751 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d4f8f-96e2-4de1-bcba-633b58076ad9 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:38.168502 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 151/304] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:38.182048 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ce7c23-c29a-4b3a-a7b8-43d10af045dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:38.183328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce7c23-c29a-4b3a-a7b8-43d10af045dc None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:38.183671 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce7c23-c29a-4b3a-a7b8-43d10af045dc None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:38.183948 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce7c23-c29a-4b3a-a7b8-43d10af045dc None admin] PATH_INFO: `/v3/users/ae0fd5d433964d0aa6993db3d36009e6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:38.191928 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7436257b-e0a2-447f-b4fa-101136f5ebe8 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:38.200045 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ce7c23-c29a-4b3a-a7b8-43d10af045dc None admin] RBAC: Authorizing `identity:get_user(user_id=ae0fd5d433964d0aa6993db3d36009e6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:38.201666 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ce7c23-c29a-4b3a-a7b8-43d10af045dc None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:38.203869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 154/305] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:38 2022] GET /identity/v3/users/ae0fd5d433964d0aa6993db3d36009e6 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:38.209156 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ce7c23-c29a-4b3a-a7b8-43d10af045dc None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:38.309314 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:38.312771 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7436257b-e0a2-447f-b4fa-101136f5ebe8 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:38.331798 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:38.342757 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ce7c23-c29a-4b3a-a7b8-43d10af045dc None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:38.355978 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ebc24f-be93-4a55-8f1e-0e8da68fe320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:38.356855 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ebc24f-be93-4a55-8f1e-0e8da68fe320 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:38.357284 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ebc24f-be93-4a55-8f1e-0e8da68fe320 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:38.357729 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ebc24f-be93-4a55-8f1e-0e8da68fe320 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:38.374169 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ebc24f-be93-4a55-8f1e-0e8da68fe320 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:38.381189 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-77ebc24f-be93-4a55-8f1e-0e8da68fe320 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:55:38.381189 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:38.381189 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:38.381189 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:38.381189 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:38.381189 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:38.381189 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:38.381189 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:38.381189 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:38.381189 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:38.381189 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:38.381189 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:38.381189 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:38.381189 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:38.381189 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:38.381189 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:38.382150 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:38.382150 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:38.382150 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:38.382150 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:38.382150 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:38.382150 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:38.382150 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:38.382150 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:38.382150 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:38.382150 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:38.382150 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:38.382150 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:38.382150 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:38.382150 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:38.382150 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:38.382150 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:38.382926 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:38.382926 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:38.382926 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:38.382926 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:38.382926 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:38.382926 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:38.382926 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:38.382926 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:38.382926 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:38.382926 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:38.382926 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:55:38.382926 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:55:38.382926 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 152/306] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:55:38.383826 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22827473-9a7d-475d-8727-c05833ad405f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:38.388561 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22827473-9a7d-475d-8727-c05833ad405f None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:38.388561 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22827473-9a7d-475d-8727-c05833ad405f None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:38.388561 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22827473-9a7d-475d-8727-c05833ad405f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:38.388561 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22827473-9a7d-475d-8727-c05833ad405f None admin] RBAC: Adding query filter params (role.id=6617379dc72a414bbf45eae46f9d74b2, user.id=ae0fd5d433964d0aa6993db3d36009e6, scope.system=all) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:38.388561 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22827473-9a7d-475d-8727-c05833ad405f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:38.389761 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22827473-9a7d-475d-8727-c05833ad405f None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:38.391260 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ebc24f-be93-4a55-8f1e-0e8da68fe320 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:38.398237 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 155/307] 10.222.0.48 () {56 vars in 1308 bytes} [Mon Aug 1 13:55:38 2022] GET /identity/v3/role_assignments?user.id=ae0fd5d433964d0aa6993db3d36009e6&scope.system=all&role.id=6617379dc72a414bbf45eae46f9d74b2 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:38.513388 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ebc24f-be93-4a55-8f1e-0e8da68fe320 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:38.553101 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eba087-e0bd-452b-8fc3-e81b3c96f3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:38.554097 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eba087-e0bd-452b-8fc3-e81b3c96f3f8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:38.554625 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eba087-e0bd-452b-8fc3-e81b3c96f3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:38.555131 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eba087-e0bd-452b-8fc3-e81b3c96f3f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:38.556080 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eba087-e0bd-452b-8fc3-e81b3c96f3f8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:38.556634 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eba087-e0bd-452b-8fc3-e81b3c96f3f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:38.558289 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eba087-e0bd-452b-8fc3-e81b3c96f3f8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:38.568746 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 153/308] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 13:55:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:38.573096 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22827473-9a7d-475d-8727-c05833ad405f None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:38.684290 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22827473-9a7d-475d-8727-c05833ad405f None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:38.720093 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbc5625-1079-46bb-bdb7-965d9d97288f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:38.720766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc5625-1079-46bb-bdb7-965d9d97288f None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:38.721049 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc5625-1079-46bb-bdb7-965d9d97288f None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:38.721327 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc5625-1079-46bb-bdb7-965d9d97288f None admin] PATH_INFO: `/v3/users/7f6b3483a62f4e4587c64b29663b856a` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:38.722501 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbc5625-1079-46bb-bdb7-965d9d97288f None admin] RBAC: Authorizing `identity:get_user(user_id=7f6b3483a62f4e4587c64b29663b856a)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:38.723925 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbc5625-1079-46bb-bdb7-965d9d97288f None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:38.725551 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 156/309] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:38 2022] GET /identity/v3/users/7f6b3483a62f4e4587c64b29663b856a => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:38.731301 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eba087-e0bd-452b-8fc3-e81b3c96f3f8 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:38.756901 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:38.837868 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eba087-e0bd-452b-8fc3-e81b3c96f3f8 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:38.877030 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5352474c-3fe0-4a09-bd65-7f6bffcbe25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:38.877710 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352474c-3fe0-4a09-bd65-7f6bffcbe25e None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:38.877994 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352474c-3fe0-4a09-bd65-7f6bffcbe25e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:38.878272 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352474c-3fe0-4a09-bd65-7f6bffcbe25e None admin] PATH_INFO: `/v3/system/users/7f6b3483a62f4e4587c64b29663b856a/roles/926b9f36c24847c582c84af904250abc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:38.881288 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5352474c-3fe0-4a09-bd65-7f6bffcbe25e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7f6b3483a62f4e4587c64b29663b856a, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:38.882674 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5352474c-3fe0-4a09-bd65-7f6bffcbe25e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:38.896777 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 154/310] 10.222.0.48 () {58 vars in 1334 bytes} [Mon Aug 1 13:55:38 2022] PUT /identity/v3/system/users/7f6b3483a62f4e4587c64b29663b856a/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:55:39.033596 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048a2e4-4102-43b3-b858-871ea3dd6d81 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:39.034243 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048a2e4-4102-43b3-b858-871ea3dd6d81 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:39.034755 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048a2e4-4102-43b3-b858-871ea3dd6d81 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:39.036687 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 157/311] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:39.043034 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0759c92-1521-42c8-bdd2-7442e007a128 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:39.043328 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0759c92-1521-42c8-bdd2-7442e007a128 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:39.043610 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0759c92-1521-42c8-bdd2-7442e007a128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:39.064193 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a0759c92-1521-42c8-bdd2-7442e007a128 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:39.065423 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a0759c92-1521-42c8-bdd2-7442e007a128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:39.134846 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 155/312] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:39.157490 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048a2e4-4102-43b3-b858-871ea3dd6d81 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:39.184414 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:39.272357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048a2e4-4102-43b3-b858-871ea3dd6d81 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:39.310107 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f536540a-287d-4e44-b77c-3753777edf4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:39.310849 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536540a-287d-4e44-b77c-3753777edf4a None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:39.311135 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536540a-287d-4e44-b77c-3753777edf4a None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:39.311413 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536540a-287d-4e44-b77c-3753777edf4a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:39.325663 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f536540a-287d-4e44-b77c-3753777edf4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:39.330931 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-f536540a-287d-4e44-b77c-3753777edf4a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 13:55:39.330931 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:39.330931 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:39.330931 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:39.330931 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:39.330931 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:39.330931 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:39.330931 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:39.330931 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:39.330931 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:39.330931 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:39.330931 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:39.330931 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:39.330931 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:39.330931 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:39.330931 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:39.331774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:39.331774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:39.331774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:39.331774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:39.331774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:39.331774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:39.331774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:39.331774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:39.331774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:39.331774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:39.331774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:39.331774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:39.331774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:39.331774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:39.331774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:39.331774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:39.333393 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:39.333393 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:39.333393 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:39.333393 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:39.333393 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:39.333393 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:39.333393 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:39.333393 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:39.333393 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:39.333393 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:39.333393 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 13:55:39.333393 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:55:39.333393 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 158/313] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:39.338262 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0759c92-1521-42c8-bdd2-7442e007a128 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:39.358724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:39.451206 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0759c92-1521-42c8-bdd2-7442e007a128 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:39.487219 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ae6df1-510a-4416-bb22-dc5629845217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:39.487840 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae6df1-510a-4416-bb22-dc5629845217 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:39.488123 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae6df1-510a-4416-bb22-dc5629845217 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:39.488403 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae6df1-510a-4416-bb22-dc5629845217 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:39.489152 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ae6df1-510a-4416-bb22-dc5629845217 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:39.489459 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ae6df1-510a-4416-bb22-dc5629845217 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:39.490825 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ae6df1-510a-4416-bb22-dc5629845217 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:39.499683 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 156/314] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 13:55:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:39.504937 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f536540a-287d-4e44-b77c-3753777edf4a None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:39.594258 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:39.615411 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f536540a-287d-4e44-b77c-3753777edf4a None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:39.652306 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ea383-1d84-4c5f-a156-84095a48b467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:39.653272 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ea383-1d84-4c5f-a156-84095a48b467 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:39.653552 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ea383-1d84-4c5f-a156-84095a48b467 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:39.653823 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ea383-1d84-4c5f-a156-84095a48b467 None admin] PATH_INFO: `/v3/users/2432796f70bc43e7917936e387d80a6a` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:39.655030 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ea383-1d84-4c5f-a156-84095a48b467 None admin] RBAC: Authorizing `identity:get_user(user_id=2432796f70bc43e7917936e387d80a6a)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:39.656469 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ea383-1d84-4c5f-a156-84095a48b467 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:39.658430 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 159/315] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:39 2022] GET /identity/v3/users/2432796f70bc43e7917936e387d80a6a => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:39.665568 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ae6df1-510a-4416-bb22-dc5629845217 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:39.752414 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:39.786663 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ae6df1-510a-4416-bb22-dc5629845217 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:39.820898 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9832cec8-94bb-4c48-a06b-03de5552a205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:39.821528 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832cec8-94bb-4c48-a06b-03de5552a205 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:39.821817 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832cec8-94bb-4c48-a06b-03de5552a205 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:39.822095 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832cec8-94bb-4c48-a06b-03de5552a205 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:39.896114 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9832cec8-94bb-4c48-a06b-03de5552a205 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:39.897638 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9832cec8-94bb-4c48-a06b-03de5552a205 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:39.976087 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:39.999597 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 157/316] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:55:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:55:40.005130 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ea383-1d84-4c5f-a156-84095a48b467 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:40.130640 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ea383-1d84-4c5f-a156-84095a48b467 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:40.170890 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8864ab9e-02dc-4f06-88fb-ebc5fd57ea01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:40.171565 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864ab9e-02dc-4f06-88fb-ebc5fd57ea01 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:40.172038 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864ab9e-02dc-4f06-88fb-ebc5fd57ea01 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:40.172434 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864ab9e-02dc-4f06-88fb-ebc5fd57ea01 None admin] PATH_INFO: `/v3/projects/05e772d8318b47469008ecb063004d71` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:40.173985 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8864ab9e-02dc-4f06-88fb-ebc5fd57ea01 None admin] RBAC: Authorizing `identity:get_project(project_id=05e772d8318b47469008ecb063004d71)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:40.175457 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8864ab9e-02dc-4f06-88fb-ebc5fd57ea01 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:40.177493 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 160/317] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:40 2022] GET /identity/v3/projects/05e772d8318b47469008ecb063004d71 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:40.183396 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9832cec8-94bb-4c48-a06b-03de5552a205 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:40.287429 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9832cec8-94bb-4c48-a06b-03de5552a205 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:40.301999 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a01b0-6e97-4264-ab1b-1546aa7e87b7 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:40.302271 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a01b0-6e97-4264-ab1b-1546aa7e87b7 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:40.302558 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a01b0-6e97-4264-ab1b-1546aa7e87b7 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:40.304180 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 161/318] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:40.309616 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57eff7c-6c70-4de6-8da0-4590cd73f578 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:40.310006 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57eff7c-6c70-4de6-8da0-4590cd73f578 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:40.310375 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57eff7c-6c70-4de6-8da0-4590cd73f578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:40.328736 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-c57eff7c-6c70-4de6-8da0-4590cd73f578 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:40.329929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-c57eff7c-6c70-4de6-8da0-4590cd73f578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:40.331109 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ffd8d-ec4f-4dca-ab2b-55b0a02ae455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:40.332009 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ffd8d-ec4f-4dca-ab2b-55b0a02ae455 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:40.332305 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ffd8d-ec4f-4dca-ab2b-55b0a02ae455 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:40.333432 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ffd8d-ec4f-4dca-ab2b-55b0a02ae455 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:40.334253 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421ffd8d-ec4f-4dca-ab2b-55b0a02ae455 None admin] RBAC: Adding query filter params (role.id=6617379dc72a414bbf45eae46f9d74b2, scope.project.id=05e772d8318b47469008ecb063004d71, user.id=2432796f70bc43e7917936e387d80a6a) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:40.334572 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421ffd8d-ec4f-4dca-ab2b-55b0a02ae455 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:40.336019 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421ffd8d-ec4f-4dca-ab2b-55b0a02ae455 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:40.343468 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 158/319] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:55:40 2022] GET /identity/v3/role_assignments?user.id=2432796f70bc43e7917936e387d80a6a&scope.project.id=05e772d8318b47469008ecb063004d71&role.id=6617379dc72a414bbf45eae46f9d74b2 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:40.344724 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:40.415052 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 162/320] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:40.441089 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ffd8d-ec4f-4dca-ab2b-55b0a02ae455 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:40.502186 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:40.508447 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1cfcbd-4f16-4f5b-a1d9-094dbc0d9138 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:40.508447 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1cfcbd-4f16-4f5b-a1d9-094dbc0d9138 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:40.508447 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1cfcbd-4f16-4f5b-a1d9-094dbc0d9138 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:40.508447 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 163/321] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:40.515111 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d160af-0494-428b-afd5-bafd5198942b None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:40.515387 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d160af-0494-428b-afd5-bafd5198942b None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:40.515723 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d160af-0494-428b-afd5-bafd5198942b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:40.539015 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-82d160af-0494-428b-afd5-bafd5198942b None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:40.540632 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-82d160af-0494-428b-afd5-bafd5198942b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:40.576605 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ffd8d-ec4f-4dca-ab2b-55b0a02ae455 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:40.616873 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825efff-d086-4ab4-91f0-98547fa8b692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:40.617527 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825efff-d086-4ab4-91f0-98547fa8b692 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:40.617820 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825efff-d086-4ab4-91f0-98547fa8b692 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:40.618104 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825efff-d086-4ab4-91f0-98547fa8b692 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:40.619276 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a825efff-d086-4ab4-91f0-98547fa8b692 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:40.620821 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a825efff-d086-4ab4-91f0-98547fa8b692 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:40.622840 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 159/322] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:55:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:40.628167 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825efff-d086-4ab4-91f0-98547fa8b692 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:40.631830 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 164/323] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:55:40.655371 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d160af-0494-428b-afd5-bafd5198942b None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:40.735662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:40.735849 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825efff-d086-4ab4-91f0-98547fa8b692 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:40.748750 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:40.760742 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d160af-0494-428b-afd5-bafd5198942b None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:40.774295 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3f4f2-2c0d-4b16-b2e0-09c083d11793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:40.774913 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3f4f2-2c0d-4b16-b2e0-09c083d11793 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:40.775197 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3f4f2-2c0d-4b16-b2e0-09c083d11793 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:40.775475 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3f4f2-2c0d-4b16-b2e0-09c083d11793 None admin] PATH_INFO: `/v3/groups` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:40.776179 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b3f4f2-2c0d-4b16-b2e0-09c083d11793 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:40.778147 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b3f4f2-2c0d-4b16-b2e0-09c083d11793 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:40.794055 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 160/324] 10.222.0.48 () {60 vars in 1134 bytes} [Mon Aug 1 13:55:40 2022] POST /identity/v3/groups => generated 226 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 13:55:40.806771 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041b325-b1f0-44ad-9ba2-de306fb209dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:40.807500 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041b325-b1f0-44ad-9ba2-de306fb209dd None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:40.807789 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041b325-b1f0-44ad-9ba2-de306fb209dd None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:40.808117 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041b325-b1f0-44ad-9ba2-de306fb209dd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:40.823856 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f041b325-b1f0-44ad-9ba2-de306fb209dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:40.829256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-f041b325-b1f0-44ad-9ba2-de306fb209dd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 13:55:40.829256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:40.829256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:40.829256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:40.829256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:40.829256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:40.829256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:40.829256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:40.829256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:40.829256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:40.829256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:40.829256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:40.829256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:40.829256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:40.829256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:40.829256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:40.830208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:40.830208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:40.830208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:40.830208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:40.830208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:40.830208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:40.830208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:40.830208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:40.830208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:40.830208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:40.830208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:40.830208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:40.830208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:40.830208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:40.830208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:40.830208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:40.833395 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:40.833395 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:40.833395 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:40.833395 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:40.833395 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:40.833395 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:40.833395 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:40.833395 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:40.833395 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:40.833395 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:40.833395 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 13:55:40.833395 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:55:40.833395 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 165/325] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:55:40.836428 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3f4f2-2c0d-4b16-b2e0-09c083d11793 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:40.947686 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3f4f2-2c0d-4b16-b2e0-09c083d11793 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:40.990269 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b4dce6-af4c-4e29-8684-c096e8468fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:40.990269 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4dce6-af4c-4e29-8684-c096e8468fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:40.990527 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4dce6-af4c-4e29-8684-c096e8468fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:40.990596 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4dce6-af4c-4e29-8684-c096e8468fb8 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:40.991945 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b4dce6-af4c-4e29-8684-c096e8468fb8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:40.991945 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b4dce6-af4c-4e29-8684-c096e8468fb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:40.993070 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b4dce6-af4c-4e29-8684-c096e8468fb8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:41.003015 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 161/326] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 13:55:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:41.009204 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041b325-b1f0-44ad-9ba2-de306fb209dd None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:41.066972 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145b7e1-dba4-41db-b0f3-a48d69972b45 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:41.067195 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145b7e1-dba4-41db-b0f3-a48d69972b45 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:41.067475 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145b7e1-dba4-41db-b0f3-a48d69972b45 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:41.070127 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 162/327] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:55:41.074904 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae024f-3755-4252-87c6-e74398b745df None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:41.075304 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae024f-3755-4252-87c6-e74398b745df None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:41.076333 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae024f-3755-4252-87c6-e74398b745df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:41.096568 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5aae024f-3755-4252-87c6-e74398b745df None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:41.098038 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5aae024f-3755-4252-87c6-e74398b745df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:41.127979 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:41.128237 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041b325-b1f0-44ad-9ba2-de306fb209dd None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:41.170310 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7916555-f0c2-49c0-8856-43833af33a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:41.170969 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7916555-f0c2-49c0-8856-43833af33a02 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:41.171249 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7916555-f0c2-49c0-8856-43833af33a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:41.171530 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7916555-f0c2-49c0-8856-43833af33a02 None admin] PATH_INFO: `/v3/users/ae0fd5d433964d0aa6993db3d36009e6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:41.172911 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7916555-f0c2-49c0-8856-43833af33a02 None admin] RBAC: Authorizing `identity:get_user(user_id=ae0fd5d433964d0aa6993db3d36009e6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:41.175201 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7916555-f0c2-49c0-8856-43833af33a02 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:41.177503 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 166/328] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:41 2022] GET /identity/v3/users/ae0fd5d433964d0aa6993db3d36009e6 => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:41.183386 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7916555-f0c2-49c0-8856-43833af33a02 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:41.187060 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 163/329] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:55:41.196548 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:41.210509 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae024f-3755-4252-87c6-e74398b745df None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:41.313947 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7916555-f0c2-49c0-8856-43833af33a02 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:41.342807 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae024f-3755-4252-87c6-e74398b745df None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:41.360710 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da41fc3-500a-41d5-a500-e933ca888a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:41.361386 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da41fc3-500a-41d5-a500-e933ca888a1d None admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:41.361675 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da41fc3-500a-41d5-a500-e933ca888a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:41.362194 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da41fc3-500a-41d5-a500-e933ca888a1d None admin] PATH_INFO: `/v3/system/users/ae0fd5d433964d0aa6993db3d36009e6/roles/6617379dc72a414bbf45eae46f9d74b2` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:41.364057 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da41fc3-500a-41d5-a500-e933ca888a1d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ae0fd5d433964d0aa6993db3d36009e6, role_id=6617379dc72a414bbf45eae46f9d74b2)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:41.365463 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da41fc3-500a-41d5-a500-e933ca888a1d None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:41.377999 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 167/330] 10.222.0.48 () {58 vars in 1334 bytes} [Mon Aug 1 13:55:41 2022] PUT /identity/v3/system/users/ae0fd5d433964d0aa6993db3d36009e6/roles/6617379dc72a414bbf45eae46f9d74b2 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:55:41.391482 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de533104-0bcd-46b1-a092-c596c47812a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:41.392163 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de533104-0bcd-46b1-a092-c596c47812a4 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:41.392447 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de533104-0bcd-46b1-a092-c596c47812a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:41.392758 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de533104-0bcd-46b1-a092-c596c47812a4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:41.410334 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de533104-0bcd-46b1-a092-c596c47812a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:41.415586 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-de533104-0bcd-46b1-a092-c596c47812a4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:55:41.415586 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:41.415586 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:41.415586 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:41.415586 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:41.415586 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:41.415586 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:41.415586 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:41.415586 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:41.415586 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:41.415586 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:41.415586 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:41.415586 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:41.415586 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:41.415586 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:41.415586 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:41.416566 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:41.416566 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:41.416566 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:41.416566 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:41.416566 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:41.416566 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:41.416566 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:41.416566 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:41.416566 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:41.416566 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:41.416566 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:41.416566 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:41.416566 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:41.416566 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:41.416566 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:41.416566 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:41.417459 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:41.417459 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:41.417459 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:41.417459 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:41.417459 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:41.417459 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:41.417459 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:41.417459 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:41.417459 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:41.417459 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:41.417459 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:55:41.417459 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:55:41.417459 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 164/331] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:55:41.423723 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da41fc3-500a-41d5-a500-e933ca888a1d None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:41.509277 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:41.548550 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da41fc3-500a-41d5-a500-e933ca888a1d None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:41.587495 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cedde93-cc43-4265-b78b-47ea4a968e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:41.588128 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedde93-cc43-4265-b78b-47ea4a968e0d None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:41.588414 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedde93-cc43-4265-b78b-47ea4a968e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:41.589248 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedde93-cc43-4265-b78b-47ea4a968e0d None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:41.589977 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cedde93-cc43-4265-b78b-47ea4a968e0d None admin] RBAC: Adding query filter params (name=member) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:41.590284 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cedde93-cc43-4265-b78b-47ea4a968e0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:41.591655 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cedde93-cc43-4265-b78b-47ea4a968e0d None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:41.600761 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 168/332] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 13:55:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:41.606886 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de533104-0bcd-46b1-a092-c596c47812a4 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:41.646192 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:41.757516 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de533104-0bcd-46b1-a092-c596c47812a4 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:41.799587 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f5513-165c-4cde-84ea-42dfbfb86aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:41.799587 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f5513-165c-4cde-84ea-42dfbfb86aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:41.799587 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f5513-165c-4cde-84ea-42dfbfb86aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:41.799587 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f5513-165c-4cde-84ea-42dfbfb86aa6 None admin] PATH_INFO: `/v3/users/7f6b3483a62f4e4587c64b29663b856a` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:41.800616 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f5513-165c-4cde-84ea-42dfbfb86aa6 None admin] RBAC: Authorizing `identity:get_user(user_id=7f6b3483a62f4e4587c64b29663b856a)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:41.802056 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f5513-165c-4cde-84ea-42dfbfb86aa6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:41.804019 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 165/333] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:41 2022] GET /identity/v3/users/7f6b3483a62f4e4587c64b29663b856a => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:41.809693 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cedde93-cc43-4265-b78b-47ea4a968e0d None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:41.953582 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cedde93-cc43-4265-b78b-47ea4a968e0d None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:41.994208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:42.000753 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc8a54-2277-46f8-bf8d-e82dad94ba75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:42.001390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc8a54-2277-46f8-bf8d-e82dad94ba75 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:42.001676 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc8a54-2277-46f8-bf8d-e82dad94ba75 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:42.001959 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc8a54-2277-46f8-bf8d-e82dad94ba75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:42.002774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc8a54-2277-46f8-bf8d-e82dad94ba75 None admin] RBAC: Adding query filter params (role.id=926b9f36c24847c582c84af904250abc, user.id=7f6b3483a62f4e4587c64b29663b856a, scope.system=all) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:42.003091 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc8a54-2277-46f8-bf8d-e82dad94ba75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:42.004559 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc8a54-2277-46f8-bf8d-e82dad94ba75 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:42.019993 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 169/334] 10.222.0.48 () {56 vars in 1308 bytes} [Mon Aug 1 13:55:41 2022] GET /identity/v3/role_assignments?user.id=7f6b3483a62f4e4587c64b29663b856a&scope.system=all&role.id=926b9f36c24847c582c84af904250abc => generated 510 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:43.344642 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c4bf9d-c952-4784-a733-2443031656c8 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:43.345155 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c4bf9d-c952-4784-a733-2443031656c8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:43.345230 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c4bf9d-c952-4784-a733-2443031656c8 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:43.346609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 166/335] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:43.362354 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4269949-e60c-498d-936a-db7337ddc8a5 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:43.362853 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4269949-e60c-498d-936a-db7337ddc8a5 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:43.363060 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4269949-e60c-498d-936a-db7337ddc8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:43.389469 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-b4269949-e60c-498d-936a-db7337ddc8a5 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:43.390685 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-b4269949-e60c-498d-936a-db7337ddc8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:43.475553 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 170/336] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:43.511938 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c4bf9d-c952-4784-a733-2443031656c8 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:43.638313 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:43.652799 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c4bf9d-c952-4784-a733-2443031656c8 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:43.689772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c3102-15f7-4624-9727-b02bd4d84388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:43.690425 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c3102-15f7-4624-9727-b02bd4d84388 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:43.690736 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c3102-15f7-4624-9727-b02bd4d84388 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:43.690984 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c3102-15f7-4624-9727-b02bd4d84388 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:43.702470 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21c3102-15f7-4624-9727-b02bd4d84388 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:43.707440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-e21c3102-15f7-4624-9727-b02bd4d84388 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:55:43.707440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:43.707440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:43.707440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:43.707440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:43.707440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:43.707440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:43.707440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:43.707440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:43.707440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:43.707440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:43.707440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:43.707440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:43.707440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:43.707440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:43.707440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:43.708312 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:43.708312 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:43.708312 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:43.708312 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:43.708312 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:43.708312 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:43.708312 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:43.708312 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:43.708312 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:43.708312 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:43.708312 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:43.708312 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:43.708312 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:43.708312 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:43.708312 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:43.708312 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:43.709111 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:43.709111 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:43.709111 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:43.709111 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:43.709111 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:43.709111 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:43.709111 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:43.709111 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:43.709111 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:43.709111 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:43.709111 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:55:43.709111 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:55:43.709111 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 167/337] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:55:43.715129 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4269949-e60c-498d-936a-db7337ddc8a5 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:43.857466 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4269949-e60c-498d-936a-db7337ddc8a5 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:43.886805 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:43.905127 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e118c-0db4-4795-b2b1-3d2c9de3cba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:43.905769 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e118c-0db4-4795-b2b1-3d2c9de3cba8 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:43.906054 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e118c-0db4-4795-b2b1-3d2c9de3cba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:43.906330 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e118c-0db4-4795-b2b1-3d2c9de3cba8 None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:43.907071 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e118c-0db4-4795-b2b1-3d2c9de3cba8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:43.907379 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e118c-0db4-4795-b2b1-3d2c9de3cba8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:43.908786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e118c-0db4-4795-b2b1-3d2c9de3cba8 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:43.920920 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 171/338] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 13:55:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:43.926277 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c3102-15f7-4624-9727-b02bd4d84388 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:43.938410 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb4409a-d474-486c-a448-f3858629f6eb None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:43.939029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb4409a-d474-486c-a448-f3858629f6eb None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:43.939509 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb4409a-d474-486c-a448-f3858629f6eb None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:43.941098 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 172/339] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:43.949787 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b28ca-5702-4df1-a49f-ae71f4cc9578 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:43.950370 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b28ca-5702-4df1-a49f-ae71f4cc9578 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:43.950867 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b28ca-5702-4df1-a49f-ae71f4cc9578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:43.984277 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-619b28ca-5702-4df1-a49f-ae71f4cc9578 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:43.985939 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-619b28ca-5702-4df1-a49f-ae71f4cc9578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:44.098896 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 173/340] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:55:44.102883 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c3102-15f7-4624-9727-b02bd4d84388 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:44.134382 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b28ca-5702-4df1-a49f-ae71f4cc9578 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:44.171239 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19726b70-3593-4bc8-8ce0-eb47e0e0cea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:44.172086 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19726b70-3593-4bc8-8ce0-eb47e0e0cea2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:44.172512 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19726b70-3593-4bc8-8ce0-eb47e0e0cea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:44.172913 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19726b70-3593-4bc8-8ce0-eb47e0e0cea2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:44.217755 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:44.283635 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b28ca-5702-4df1-a49f-ae71f4cc9578 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:44.302511 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:44.346393 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df011c18-fceb-422f-9d13-c5d818467fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:44.347539 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df011c18-fceb-422f-9d13-c5d818467fd1 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:44.348912 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df011c18-fceb-422f-9d13-c5d818467fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:44.349988 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df011c18-fceb-422f-9d13-c5d818467fd1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:44.362466 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19726b70-3593-4bc8-8ce0-eb47e0e0cea2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:44.364594 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19726b70-3593-4bc8-8ce0-eb47e0e0cea2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:44.373757 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df011c18-fceb-422f-9d13-c5d818467fd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:44.384259 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-df011c18-fceb-422f-9d13-c5d818467fd1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:55:44.384259 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:44.384259 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:44.384259 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:44.384259 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:44.384259 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:44.384259 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:44.384259 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:44.384259 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:44.384259 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:44.384259 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:44.384259 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:44.384259 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:44.384259 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:44.384259 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:44.384259 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:44.385209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:44.385209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:44.385209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:44.385209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:44.385209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:44.385209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:44.385209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:44.385209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:44.385209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:44.385209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:44.385209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:44.385209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:44.385209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:44.385209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:44.385209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:44.385209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:44.386093 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:44.386093 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:44.386093 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:44.386093 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:44.386093 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:44.386093 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:44.386093 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:44.386093 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:44.386093 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:44.386093 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:44.386093 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:55:44.386093 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:55:44.386093 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 174/341] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 13:55:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:55:44.386748 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf505ca-546a-496e-a4fa-44e72c286476 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:44.387626 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf505ca-546a-496e-a4fa-44e72c286476 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:44.387626 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf505ca-546a-496e-a4fa-44e72c286476 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:44.391524 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 175/342] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:44.395058 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf505ca-546a-496e-a4fa-44e72c286476 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:44.556836 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 168/343] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:55:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 631 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:55:44.559091 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39390a06-0e55-44f3-9611-414266eaa11f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:44.559609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39390a06-0e55-44f3-9611-414266eaa11f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:44.560076 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf505ca-546a-496e-a4fa-44e72c286476 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:44.560226 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39390a06-0e55-44f3-9611-414266eaa11f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:44.593526 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-39390a06-0e55-44f3-9611-414266eaa11f None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:44.594961 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-39390a06-0e55-44f3-9611-414266eaa11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:44.605274 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c357be2-d302-4099-b67a-f9f0168a17d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:44.605908 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c357be2-d302-4099-b67a-f9f0168a17d5 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:44.606197 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c357be2-d302-4099-b67a-f9f0168a17d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:44.606476 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c357be2-d302-4099-b67a-f9f0168a17d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:44.607209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c357be2-d302-4099-b67a-f9f0168a17d5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:44.607522 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c357be2-d302-4099-b67a-f9f0168a17d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:44.608959 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c357be2-d302-4099-b67a-f9f0168a17d5 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:44.620759 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 176/344] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:55:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:44.622385 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4341c4a-cb24-4b02-9d14-35ef996f3370 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:44.622880 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4341c4a-cb24-4b02-9d14-35ef996f3370 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:44.623361 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4341c4a-cb24-4b02-9d14-35ef996f3370 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:44.625228 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 177/345] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:44.626220 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4341c4a-cb24-4b02-9d14-35ef996f3370 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:44.668685 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:44.670318 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:44.699955 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 169/346] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:44.701486 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15af79f-78ef-4f6f-848c-e45e4a1cbee3 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:44.701775 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15af79f-78ef-4f6f-848c-e45e4a1cbee3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:44.702062 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15af79f-78ef-4f6f-848c-e45e4a1cbee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:44.720778 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a15af79f-78ef-4f6f-848c-e45e4a1cbee3 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:44.721977 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a15af79f-78ef-4f6f-848c-e45e4a1cbee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:44.762334 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4341c4a-cb24-4b02-9d14-35ef996f3370 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:44.811571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae69c3f-83f1-46fa-b201-377614440227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:44.812161 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae69c3f-83f1-46fa-b201-377614440227 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:44.812449 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae69c3f-83f1-46fa-b201-377614440227 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:44.815577 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae69c3f-83f1-46fa-b201-377614440227 None admin] PATH_INFO: `/v3/projects/516ee04b71684e5885fa9a08b821fb87` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:44.815577 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae69c3f-83f1-46fa-b201-377614440227 None admin] RBAC: Authorizing `identity:get_project(project_id=516ee04b71684e5885fa9a08b821fb87)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:44.818128 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae69c3f-83f1-46fa-b201-377614440227 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:44.819719 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 170/347] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:44.820932 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15af79f-78ef-4f6f-848c-e45e4a1cbee3 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:44.824613 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 178/348] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:44 2022] GET /identity/v3/projects/516ee04b71684e5885fa9a08b821fb87 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:44.826234 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae69c3f-83f1-46fa-b201-377614440227 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:44.947896 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae69c3f-83f1-46fa-b201-377614440227 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:44.949952 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:44.969556 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15af79f-78ef-4f6f-848c-e45e4a1cbee3 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:44.985368 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:44.992443 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb3815-d5c1-44d0-b321-c34b77f39ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:44.993145 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb3815-d5c1-44d0-b321-c34b77f39ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:44.993432 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb3815-d5c1-44d0-b321-c34b77f39ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:44.993710 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb3815-d5c1-44d0-b321-c34b77f39ba4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:45.006580 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cb3815-d5c1-44d0-b321-c34b77f39ba4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:45.012626 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-49cb3815-d5c1-44d0-b321-c34b77f39ba4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 13:55:45.012626 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:45.012626 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:45.012626 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:45.012626 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:45.012626 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:45.012626 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:45.012626 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:45.012626 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:45.012626 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:45.012626 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:45.012626 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:45.012626 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:45.012626 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:45.012626 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:45.012626 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:45.013628 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:45.013628 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:45.013628 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:45.013628 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:45.013628 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:45.013628 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:45.013628 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:45.013628 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:45.013628 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:45.013628 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:45.013628 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:45.013628 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:45.013628 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:45.013628 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:45.013628 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:45.013628 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:45.014445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:45.014445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:45.014445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:45.014445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:45.014445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:45.014445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:45.014445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:45.014445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:45.014445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:45.014445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:45.014445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 13:55:45.014445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:55:45.016091 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 179/349] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:45.017258 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb3815-d5c1-44d0-b321-c34b77f39ba4 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:45.019833 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1cab4-c05e-4376-9705-54b1f35fc428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:45.021083 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1cab4-c05e-4376-9705-54b1f35fc428 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:45.022586 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1cab4-c05e-4376-9705-54b1f35fc428 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:45.023158 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1cab4-c05e-4376-9705-54b1f35fc428 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:45.133438 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b1cab4-c05e-4376-9705-54b1f35fc428 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:45.135608 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b1cab4-c05e-4376-9705-54b1f35fc428 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:45.157909 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb3815-d5c1-44d0-b321-c34b77f39ba4 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:45.194766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30907ad-8399-4338-ae04-1f9a46388247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:45.195516 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30907ad-8399-4338-ae04-1f9a46388247 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:45.195873 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30907ad-8399-4338-ae04-1f9a46388247 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:45.196288 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30907ad-8399-4338-ae04-1f9a46388247 None admin] PATH_INFO: `/v3/groups/158f3c1b59904f1f93ecd8987d832191` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:45.202796 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30907ad-8399-4338-ae04-1f9a46388247 None admin] RBAC: Authorizing `identity:get_group(group_id=158f3c1b59904f1f93ecd8987d832191)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:45.204365 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30907ad-8399-4338-ae04-1f9a46388247 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:45.211061 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 180/350] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 13:55:45 2022] GET /identity/v3/groups/158f3c1b59904f1f93ecd8987d832191 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:45.211982 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30907ad-8399-4338-ae04-1f9a46388247 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:45.242213 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:45.251146 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 171/351] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:55:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:55:45.252265 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1cab4-c05e-4376-9705-54b1f35fc428 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:45.266214 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:45.341623 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30907ad-8399-4338-ae04-1f9a46388247 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:45.376470 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1cab4-c05e-4376-9705-54b1f35fc428 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:45.390244 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa47588-4abc-48d9-b39a-4c5aa4f47504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:45.390864 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa47588-4abc-48d9-b39a-4c5aa4f47504 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:45.391116 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa47588-4abc-48d9-b39a-4c5aa4f47504 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:45.391403 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa47588-4abc-48d9-b39a-4c5aa4f47504 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:45.404846 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa47588-4abc-48d9-b39a-4c5aa4f47504 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:45.412512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-bfa47588-4abc-48d9-b39a-4c5aa4f47504 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:55:45.412512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:45.412512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:45.412512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:45.412512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:45.412512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:45.412512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:45.412512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:45.412512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:45.412512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:45.412512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:45.412512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:45.412512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:45.412512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:45.412512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:45.412512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:45.413586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:45.413586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:45.413586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:45.413586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:45.413586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:45.413586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:45.413586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:45.413586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:45.413586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:45.413586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:45.413586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:45.413586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:45.413586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:45.413586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:45.413586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:45.413586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:45.414579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:45.414579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:45.414579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:45.414579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:45.414579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:45.414579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:45.414579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:45.414579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:45.414579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:45.414579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:45.414579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:55:45.414579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:55:45.416553 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 181/352] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:45.418448 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa47588-4abc-48d9-b39a-4c5aa4f47504 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:45.425669 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19522ba-ae99-432d-a89a-2faae5b6198b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:45.426312 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19522ba-ae99-432d-a89a-2faae5b6198b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:45.426596 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19522ba-ae99-432d-a89a-2faae5b6198b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:45.426878 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19522ba-ae99-432d-a89a-2faae5b6198b None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:45.428010 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19522ba-ae99-432d-a89a-2faae5b6198b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:45.428348 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19522ba-ae99-432d-a89a-2faae5b6198b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:45.429796 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19522ba-ae99-432d-a89a-2faae5b6198b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:45.441773 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 172/353] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 13:55:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:45.442783 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19522ba-ae99-432d-a89a-2faae5b6198b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:45.543111 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:45.547645 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:45.562155 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa47588-4abc-48d9-b39a-4c5aa4f47504 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:45.584623 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19522ba-ae99-432d-a89a-2faae5b6198b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:45.602746 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850ee71-0a84-4745-a529-7b02ece59284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:45.603459 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850ee71-0a84-4745-a529-7b02ece59284 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:45.603741 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850ee71-0a84-4745-a529-7b02ece59284 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:45.604063 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850ee71-0a84-4745-a529-7b02ece59284 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:45.605430 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6850ee71-0a84-4745-a529-7b02ece59284 None admin] RBAC: Adding query filter params (group.id=158f3c1b59904f1f93ecd8987d832191, role.id=926b9f36c24847c582c84af904250abc, scope.project.id=516ee04b71684e5885fa9a08b821fb87) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:45.606277 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6850ee71-0a84-4745-a529-7b02ece59284 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:45.607845 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6850ee71-0a84-4745-a529-7b02ece59284 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:45.613783 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 182/354] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 1 13:55:45 2022] GET /identity/v3/role_assignments?group.id=158f3c1b59904f1f93ecd8987d832191&scope.project.id=516ee04b71684e5885fa9a08b821fb87&role.id=926b9f36c24847c582c84af904250abc => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:45.615230 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab3308-77b9-494d-a3aa-f6cc7865e3cd None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:45.615595 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab3308-77b9-494d-a3aa-f6cc7865e3cd None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:45.615877 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab3308-77b9-494d-a3aa-f6cc7865e3cd None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:45.617334 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 183/355] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:45.622900 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab3308-77b9-494d-a3aa-f6cc7865e3cd None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:45.629183 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1983f-b3f8-404f-b3b8-4b832bec5a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:45.629869 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1983f-b3f8-404f-b3b8-4b832bec5a4d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:45.630158 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1983f-b3f8-404f-b3b8-4b832bec5a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:45.630438 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1983f-b3f8-404f-b3b8-4b832bec5a4d None admin] PATH_INFO: `/v3/projects/516ee04b71684e5885fa9a08b821fb87` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:45.631820 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c1983f-b3f8-404f-b3b8-4b832bec5a4d None admin] RBAC: Authorizing `identity:get_project(project_id=516ee04b71684e5885fa9a08b821fb87)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:45.636838 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c1983f-b3f8-404f-b3b8-4b832bec5a4d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:45.638560 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 173/356] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:45 2022] GET /identity/v3/projects/516ee04b71684e5885fa9a08b821fb87 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:45.640321 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1983f-b3f8-404f-b3b8-4b832bec5a4d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:45.772147 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab3308-77b9-494d-a3aa-f6cc7865e3cd None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:45.777159 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1983f-b3f8-404f-b3b8-4b832bec5a4d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:45.823458 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09af58b-31fb-4fcf-a216-9774fae1a8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:45.824092 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09af58b-31fb-4fcf-a216-9774fae1a8ea None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:45.824375 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09af58b-31fb-4fcf-a216-9774fae1a8ea None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:45.824700 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09af58b-31fb-4fcf-a216-9774fae1a8ea None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:45.825433 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09af58b-31fb-4fcf-a216-9774fae1a8ea None admin] RBAC: Adding query filter params (name=member) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:45.825739 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09af58b-31fb-4fcf-a216-9774fae1a8ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:45.827122 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09af58b-31fb-4fcf-a216-9774fae1a8ea None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:45.829271 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d9dee-5721-41c9-b689-25c5482fd94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:45.830036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d9dee-5721-41c9-b689-25c5482fd94b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:45.830392 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d9dee-5721-41c9-b689-25c5482fd94b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:45.830741 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d9dee-5721-41c9-b689-25c5482fd94b None admin] PATH_INFO: `/v3/users/ae0fd5d433964d0aa6993db3d36009e6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:45.832074 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7d9dee-5721-41c9-b689-25c5482fd94b None admin] RBAC: Authorizing `identity:get_user(user_id=ae0fd5d433964d0aa6993db3d36009e6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:45.833652 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7d9dee-5721-41c9-b689-25c5482fd94b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:45.840569 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 184/358] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 13:55:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:45.840569 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8cc3e9-16cb-4877-9d7e-c036f7b17c6f None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:45.840569 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8cc3e9-16cb-4877-9d7e-c036f7b17c6f None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:45.840569 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8cc3e9-16cb-4877-9d7e-c036f7b17c6f None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:45.840953 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 174/358] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:55:45 2022] GET /identity/v3/users/ae0fd5d433964d0aa6993db3d36009e6 => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:45.841023 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 185/359] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:45.842434 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa128f-1590-4388-8110-79e9b494fe49 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:45.842792 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa128f-1590-4388-8110-79e9b494fe49 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:45.843023 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa128f-1590-4388-8110-79e9b494fe49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:45.846003 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8cc3e9-16cb-4877-9d7e-c036f7b17c6f None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:45.863323 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:45.865162 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-3bfa128f-1590-4388-8110-79e9b494fe49 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:45.866938 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3bfa128f-1590-4388-8110-79e9b494fe49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:45.868859 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:45.957298 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 175/360] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:45.958625 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa128f-1590-4388-8110-79e9b494fe49 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:45.993855 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8cc3e9-16cb-4877-9d7e-c036f7b17c6f None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:46.050115 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0097ad67-ce9a-46e2-b2e0-f71e1f29fc40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:46.050776 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097ad67-ce9a-46e2-b2e0-f71e1f29fc40 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:46.051058 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097ad67-ce9a-46e2-b2e0-f71e1f29fc40 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:46.051335 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097ad67-ce9a-46e2-b2e0-f71e1f29fc40 None admin] PATH_INFO: `/v3/groups/158f3c1b59904f1f93ecd8987d832191` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:46.052632 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0097ad67-ce9a-46e2-b2e0-f71e1f29fc40 None admin] RBAC: Authorizing `identity:get_group(group_id=158f3c1b59904f1f93ecd8987d832191)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:46.054537 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0097ad67-ce9a-46e2-b2e0-f71e1f29fc40 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:46.060542 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 186/361] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 13:55:45 2022] GET /identity/v3/groups/158f3c1b59904f1f93ecd8987d832191 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:46.060542 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892851bc-3e09-48b6-8f7e-d004fef15459 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:46.060542 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892851bc-3e09-48b6-8f7e-d004fef15459 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:46.060542 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892851bc-3e09-48b6-8f7e-d004fef15459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:46.083446 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-892851bc-3e09-48b6-8f7e-d004fef15459 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:46.085118 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-892851bc-3e09-48b6-8f7e-d004fef15459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:46.090347 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa128f-1590-4388-8110-79e9b494fe49 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:46.140598 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ef832-9169-422a-bc04-4e1b625027a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:46.141628 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ef832-9169-422a-bc04-4e1b625027a9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:46.141917 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ef832-9169-422a-bc04-4e1b625027a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:46.142195 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ef832-9169-422a-bc04-4e1b625027a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:46.153949 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:46.172871 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 187/362] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:46.173822 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892851bc-3e09-48b6-8f7e-d004fef15459 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:46.188606 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:46.221540 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ef832-9169-422a-bc04-4e1b625027a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:46.223047 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ef832-9169-422a-bc04-4e1b625027a9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:46.311631 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892851bc-3e09-48b6-8f7e-d004fef15459 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:46.350839 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 176/363] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:55:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:55:46.351702 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ef832-9169-422a-bc04-4e1b625027a9 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:46.363632 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c168e-a523-404d-8647-2eae5ceb69ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:46.364265 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c168e-a523-404d-8647-2eae5ceb69ae None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:46.364607 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c168e-a523-404d-8647-2eae5ceb69ae None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:46.364890 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c168e-a523-404d-8647-2eae5ceb69ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:46.365756 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502c168e-a523-404d-8647-2eae5ceb69ae None admin] RBAC: Adding query filter params (role.id=6617379dc72a414bbf45eae46f9d74b2, user.id=ae0fd5d433964d0aa6993db3d36009e6, scope.system=all) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:46.366075 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502c168e-a523-404d-8647-2eae5ceb69ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:46.367564 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502c168e-a523-404d-8647-2eae5ceb69ae None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:46.384083 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 188/364] 10.222.0.48 () {56 vars in 1308 bytes} [Mon Aug 1 13:55:46 2022] GET /identity/v3/role_assignments?user.id=ae0fd5d433964d0aa6993db3d36009e6&scope.system=all&role.id=6617379dc72a414bbf45eae46f9d74b2 => generated 510 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:46.384645 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c168e-a523-404d-8647-2eae5ceb69ae None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:46.405747 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:46.460909 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:46.486031 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ef832-9169-422a-bc04-4e1b625027a9 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:46.501626 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c168e-a523-404d-8647-2eae5ceb69ae None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:46.542474 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac80d4f7-ca68-41c7-aed9-e36f0bc41ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:46.543132 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80d4f7-ca68-41c7-aed9-e36f0bc41ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:46.543414 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80d4f7-ca68-41c7-aed9-e36f0bc41ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:46.543690 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80d4f7-ca68-41c7-aed9-e36f0bc41ff1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:46.565829 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac80d4f7-ca68-41c7-aed9-e36f0bc41ff1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:46.566555 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b93f0-8fcc-4075-8f71-b2d12a9d3da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:46.567474 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b93f0-8fcc-4075-8f71-b2d12a9d3da8 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:46.567961 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b93f0-8fcc-4075-8f71-b2d12a9d3da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:46.568519 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b93f0-8fcc-4075-8f71-b2d12a9d3da8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:46.569592 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b93f0-8fcc-4075-8f71-b2d12a9d3da8 None admin] RBAC: Adding query filter params (group.id=158f3c1b59904f1f93ecd8987d832191, role.id=47e170213d8049a28f8b8a0d736fdf6b, scope.project.id=516ee04b71684e5885fa9a08b821fb87) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:46.570110 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b93f0-8fcc-4075-8f71-b2d12a9d3da8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:46.570869 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-ac80d4f7-ca68-41c7-aed9-e36f0bc41ff1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:55:46.570869 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:46.570869 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:46.570869 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:46.570869 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:46.570869 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:46.570869 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:46.570869 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:46.570869 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:46.570869 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:46.570869 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:46.570869 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:46.570869 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:46.570869 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:46.570869 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:46.570869 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:46.571765 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:46.571765 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:46.571765 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:46.571765 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:46.571765 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:46.571765 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:46.571765 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:46.571765 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:46.571765 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:46.571765 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:46.571765 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:46.571765 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:46.571765 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:46.571765 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:46.571765 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:46.571765 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:46.573609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:46.573609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:46.573609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:46.573609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:46.573609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:46.573609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:46.573609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:46.573609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:46.573609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:46.573609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:46.573609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:55:46.573609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:55:46.573609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 177/365] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 13:55:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:46.573609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac80d4f7-ca68-41c7-aed9-e36f0bc41ff1 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:46.575834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b93f0-8fcc-4075-8f71-b2d12a9d3da8 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:46.585632 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 189/366] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 1 13:55:46 2022] GET /identity/v3/role_assignments?group.id=158f3c1b59904f1f93ecd8987d832191&scope.project.id=516ee04b71684e5885fa9a08b821fb87&role.id=47e170213d8049a28f8b8a0d736fdf6b => generated 260 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:46.587348 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b93f0-8fcc-4075-8f71-b2d12a9d3da8 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:46.712851 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:46.718073 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac80d4f7-ca68-41c7-aed9-e36f0bc41ff1 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:46.736165 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b93f0-8fcc-4075-8f71-b2d12a9d3da8 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:46.768890 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf216e82-4bc4-4a2e-89dd-d492efe3fa1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:46.769874 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf216e82-4bc4-4a2e-89dd-d492efe3fa1a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:46.770148 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf216e82-4bc4-4a2e-89dd-d492efe3fa1a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:46.770430 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf216e82-4bc4-4a2e-89dd-d492efe3fa1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:46.779322 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:46.786969 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3197ea5e-a4ae-401c-8ca6-dd0e7e1147d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:46.787640 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197ea5e-a4ae-401c-8ca6-dd0e7e1147d4 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:46.787935 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197ea5e-a4ae-401c-8ca6-dd0e7e1147d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:46.788220 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197ea5e-a4ae-401c-8ca6-dd0e7e1147d4 None admin] PATH_INFO: `/v3/projects/05e772d8318b47469008ecb063004d71` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:46.788909 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf216e82-4bc4-4a2e-89dd-d492efe3fa1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:46.789526 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3197ea5e-a4ae-401c-8ca6-dd0e7e1147d4 None admin] RBAC: Authorizing `identity:get_project(project_id=05e772d8318b47469008ecb063004d71)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:46.791050 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3197ea5e-a4ae-401c-8ca6-dd0e7e1147d4 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:46.793102 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 190/367] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:46 2022] GET /identity/v3/projects/05e772d8318b47469008ecb063004d71 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:46.793898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3197ea5e-a4ae-401c-8ca6-dd0e7e1147d4 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:46.794349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-cf216e82-4bc4-4a2e-89dd-d492efe3fa1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:55:46.794349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:46.794349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:46.794349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:46.794349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:46.794349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:46.794349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:46.794349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:46.794349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:46.794349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:46.794349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:46.794349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:46.794349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:46.794349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:46.794349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:46.794349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:46.795258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:46.795258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:46.795258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:46.795258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:46.795258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:46.795258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:46.795258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:46.795258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:46.795258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:46.795258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:46.795258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:46.795258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:46.795258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:46.795258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:46.795258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:46.795258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:46.796610 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:46.796610 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:46.796610 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:46.796610 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:46.796610 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:46.796610 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:46.796610 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:46.796610 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:46.796610 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:46.796610 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:46.796610 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:55:46.796610 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:55:46.796610 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 178/368] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 13:55:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:46.801762 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf216e82-4bc4-4a2e-89dd-d492efe3fa1a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:46.941680 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3197ea5e-a4ae-401c-8ca6-dd0e7e1147d4 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:46.951744 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf216e82-4bc4-4a2e-89dd-d492efe3fa1a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:46.993203 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304081cf-7149-4a61-9d1a-c35754994513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:46.994139 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304081cf-7149-4a61-9d1a-c35754994513 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:46.995141 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304081cf-7149-4a61-9d1a-c35754994513 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:46.997632 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304081cf-7149-4a61-9d1a-c35754994513 None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:46.998674 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304081cf-7149-4a61-9d1a-c35754994513 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:46.999249 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304081cf-7149-4a61-9d1a-c35754994513 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:47.002521 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304081cf-7149-4a61-9d1a-c35754994513 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:47.003729 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb27ee7-7fb0-4e2b-a592-72bd92f6d6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:47.004400 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb27ee7-7fb0-4e2b-a592-72bd92f6d6a2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:47.004745 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb27ee7-7fb0-4e2b-a592-72bd92f6d6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:47.005023 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb27ee7-7fb0-4e2b-a592-72bd92f6d6a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:47.005751 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb27ee7-7fb0-4e2b-a592-72bd92f6d6a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:47.006060 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb27ee7-7fb0-4e2b-a592-72bd92f6d6a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:47.007442 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb27ee7-7fb0-4e2b-a592-72bd92f6d6a2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:47.014766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 191/369] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:55:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:47.014766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304081cf-7149-4a61-9d1a-c35754994513 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:47.025553 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 179/370] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:47.028400 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb27ee7-7fb0-4e2b-a592-72bd92f6d6a2 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:47.068367 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:47.110778 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:47.169249 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb27ee7-7fb0-4e2b-a592-72bd92f6d6a2 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:47.170033 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304081cf-7149-4a61-9d1a-c35754994513 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:47.204339 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73d823f-01a3-4750-bb6b-40e3c4ee01bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:47.205117 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d823f-01a3-4750-bb6b-40e3c4ee01bd None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:47.205507 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d823f-01a3-4750-bb6b-40e3c4ee01bd None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:47.205786 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d823f-01a3-4750-bb6b-40e3c4ee01bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:47.207215 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9086b9b6-4f38-41cf-9d10-650a13639a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:47.207866 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086b9b6-4f38-41cf-9d10-650a13639a15 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:47.208150 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086b9b6-4f38-41cf-9d10-650a13639a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:47.208566 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086b9b6-4f38-41cf-9d10-650a13639a15 None admin] PATH_INFO: `/v3/groups/158f3c1b59904f1f93ecd8987d832191` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:47.209837 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9086b9b6-4f38-41cf-9d10-650a13639a15 None admin] RBAC: Authorizing `identity:get_group(group_id=158f3c1b59904f1f93ecd8987d832191)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:47.211296 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9086b9b6-4f38-41cf-9d10-650a13639a15 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:47.214448 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 192/371] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 13:55:47 2022] GET /identity/v3/groups/158f3c1b59904f1f93ecd8987d832191 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:47.215478 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9086b9b6-4f38-41cf-9d10-650a13639a15 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:47.304309 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73d823f-01a3-4750-bb6b-40e3c4ee01bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:47.310708 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73d823f-01a3-4750-bb6b-40e3c4ee01bd None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:47.337172 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:47.360437 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9086b9b6-4f38-41cf-9d10-650a13639a15 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:47.394058 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:47.403758 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70436da-ce4c-407f-acdc-557cac6fa169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:47.405011 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70436da-ce4c-407f-acdc-557cac6fa169 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:47.405011 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70436da-ce4c-407f-acdc-557cac6fa169 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:47.405180 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70436da-ce4c-407f-acdc-557cac6fa169 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:47.451946 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 180/372] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:55:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:55:47.453422 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73d823f-01a3-4750-bb6b-40e3c4ee01bd None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:47.497325 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70436da-ce4c-407f-acdc-557cac6fa169 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:47.499480 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70436da-ce4c-407f-acdc-557cac6fa169 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:47.571954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73d823f-01a3-4750-bb6b-40e3c4ee01bd None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:47.593381 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:47.610473 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc6f5a-31df-4123-912d-efc453a1fa35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:47.611108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc6f5a-31df-4123-912d-efc453a1fa35 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:47.611393 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc6f5a-31df-4123-912d-efc453a1fa35 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:47.611671 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc6f5a-31df-4123-912d-efc453a1fa35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:47.612473 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fc6f5a-31df-4123-912d-efc453a1fa35 None admin] RBAC: Adding query filter params (group.id=158f3c1b59904f1f93ecd8987d832191, role.id=926b9f36c24847c582c84af904250abc, scope.project.id=05e772d8318b47469008ecb063004d71) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:47.613669 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fc6f5a-31df-4123-912d-efc453a1fa35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:47.615122 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fc6f5a-31df-4123-912d-efc453a1fa35 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:47.627093 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 181/373] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 1 13:55:47 2022] GET /identity/v3/role_assignments?group.id=158f3c1b59904f1f93ecd8987d832191&scope.project.id=05e772d8318b47469008ecb063004d71&role.id=926b9f36c24847c582c84af904250abc => generated 260 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:47.627093 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc6f5a-31df-4123-912d-efc453a1fa35 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:47.638651 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 193/374] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:55:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:55:47.645534 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70436da-ce4c-407f-acdc-557cac6fa169 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:47.706861 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:47.790185 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc6f5a-31df-4123-912d-efc453a1fa35 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:47.834717 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e7dff6-fbac-431c-973f-67f269d49789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:47.835351 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7dff6-fbac-431c-973f-67f269d49789 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:47.835640 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7dff6-fbac-431c-973f-67f269d49789 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:47.835915 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7dff6-fbac-431c-973f-67f269d49789 None admin] PATH_INFO: `/v3/projects/05e772d8318b47469008ecb063004d71` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:47.837464 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e7dff6-fbac-431c-973f-67f269d49789 None admin] RBAC: Authorizing `identity:get_project(project_id=05e772d8318b47469008ecb063004d71)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:47.839737 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e7dff6-fbac-431c-973f-67f269d49789 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:47.842466 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 182/375] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:47 2022] GET /identity/v3/projects/05e772d8318b47469008ecb063004d71 => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:47.848630 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e7dff6-fbac-431c-973f-67f269d49789 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:47.870398 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70436da-ce4c-407f-acdc-557cac6fa169 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:47.910303 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac8eae-4d79-4ba2-84e3-ae5a1d2e08b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:47.911350 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac8eae-4d79-4ba2-84e3-ae5a1d2e08b1 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:47.911686 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:47.912059 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac8eae-4d79-4ba2-84e3-ae5a1d2e08b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:47.912575 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac8eae-4d79-4ba2-84e3-ae5a1d2e08b1 None admin] PATH_INFO: `/v3/projects/b30b7ebe5b2d4e7c8f161e5b78a7a0df` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:47.914242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ac8eae-4d79-4ba2-84e3-ae5a1d2e08b1 None admin] RBAC: Authorizing `identity:get_project(project_id=b30b7ebe5b2d4e7c8f161e5b78a7a0df)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:47.915980 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ac8eae-4d79-4ba2-84e3-ae5a1d2e08b1 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:47.918327 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 194/376] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:47 2022] GET /identity/v3/projects/b30b7ebe5b2d4e7c8f161e5b78a7a0df => generated 344 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:47.926110 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac8eae-4d79-4ba2-84e3-ae5a1d2e08b1 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:47.970153 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e7dff6-fbac-431c-973f-67f269d49789 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:48.008278 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedba460-5cc2-4382-9b19-cc757640c1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:48.008969 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedba460-5cc2-4382-9b19-cc757640c1e9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:48.009302 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedba460-5cc2-4382-9b19-cc757640c1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:48.009588 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedba460-5cc2-4382-9b19-cc757640c1e9 None admin] PATH_INFO: `/v3/groups/158f3c1b59904f1f93ecd8987d832191` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:48.011287 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedba460-5cc2-4382-9b19-cc757640c1e9 None admin] RBAC: Authorizing `identity:get_group(group_id=158f3c1b59904f1f93ecd8987d832191)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:48.012764 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedba460-5cc2-4382-9b19-cc757640c1e9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:48.014843 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 183/377] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 13:55:47 2022] GET /identity/v3/groups/158f3c1b59904f1f93ecd8987d832191 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:48.016107 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7e5d8-dd21-42d0-8956-3c28b59a36f7 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:48.016395 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7e5d8-dd21-42d0-8956-3c28b59a36f7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:48.016754 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7e5d8-dd21-42d0-8956-3c28b59a36f7 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:48.018152 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 184/378] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:48.023022 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7e5d8-dd21-42d0-8956-3c28b59a36f7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:48.042613 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac8eae-4d79-4ba2-84e3-ae5a1d2e08b1 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:48.047696 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:48.082975 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26498d0-e0fe-40dd-a309-497369f02d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:48.084104 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26498d0-e0fe-40dd-a309-497369f02d7f None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:48.084633 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26498d0-e0fe-40dd-a309-497369f02d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:48.085356 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26498d0-e0fe-40dd-a309-497369f02d7f None admin] PATH_INFO: `/v3/groups/f73bbef49520470cb75b98a774a1bcfe` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:48.093603 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26498d0-e0fe-40dd-a309-497369f02d7f None admin] RBAC: Authorizing `identity:get_group(group_id=f73bbef49520470cb75b98a774a1bcfe)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:48.095418 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26498d0-e0fe-40dd-a309-497369f02d7f None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:48.102336 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 195/379] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 13:55:47 2022] GET /identity/v3/groups/f73bbef49520470cb75b98a774a1bcfe => generated 229 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:48.103987 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018ac60-57b3-4f1d-b4ef-056f39f8f1d7 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:48.104519 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018ac60-57b3-4f1d-b4ef-056f39f8f1d7 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:48.105035 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018ac60-57b3-4f1d-b4ef-056f39f8f1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:48.124759 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-c018ac60-57b3-4f1d-b4ef-056f39f8f1d7 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:48.126406 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-c018ac60-57b3-4f1d-b4ef-056f39f8f1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:48.136527 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7e5d8-dd21-42d0-8956-3c28b59a36f7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:48.159244 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:48.176585 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d123dd04-daf6-46b5-bb17-88cf2b85d7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:48.177264 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123dd04-daf6-46b5-bb17-88cf2b85d7fa None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:48.177490 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123dd04-daf6-46b5-bb17-88cf2b85d7fa None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:48.177765 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123dd04-daf6-46b5-bb17-88cf2b85d7fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:48.179595 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d123dd04-daf6-46b5-bb17-88cf2b85d7fa None admin] RBAC: Adding query filter params (group.id=158f3c1b59904f1f93ecd8987d832191, role.id=47e170213d8049a28f8b8a0d736fdf6b, scope.project.id=05e772d8318b47469008ecb063004d71) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:48.179922 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d123dd04-daf6-46b5-bb17-88cf2b85d7fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:48.181435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d123dd04-daf6-46b5-bb17-88cf2b85d7fa None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:48.187666 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 185/380] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 1 13:55:48 2022] GET /identity/v3/role_assignments?group.id=158f3c1b59904f1f93ecd8987d832191&scope.project.id=05e772d8318b47469008ecb063004d71&role.id=47e170213d8049a28f8b8a0d736fdf6b => generated 260 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:48.188720 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d123dd04-daf6-46b5-bb17-88cf2b85d7fa None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:48.207378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 196/381] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:48.232759 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c018ac60-57b3-4f1d-b4ef-056f39f8f1d7 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:48.308650 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d123dd04-daf6-46b5-bb17-88cf2b85d7fa None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:48.341040 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:48.347901 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fa1e89-8e54-4af8-ba02-5bc7284bcdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:48.348574 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa1e89-8e54-4af8-ba02-5bc7284bcdb4 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:48.348862 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa1e89-8e54-4af8-ba02-5bc7284bcdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:48.349139 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa1e89-8e54-4af8-ba02-5bc7284bcdb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:48.349953 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fa1e89-8e54-4af8-ba02-5bc7284bcdb4 None admin] RBAC: Adding query filter params (group.id=f73bbef49520470cb75b98a774a1bcfe, role.id=58f91e3a13664dabb18c6227a1658fa6, scope.project.id=b30b7ebe5b2d4e7c8f161e5b78a7a0df) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:48.350345 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fa1e89-8e54-4af8-ba02-5bc7284bcdb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:48.351810 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fa1e89-8e54-4af8-ba02-5bc7284bcdb4 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:48.355957 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c018ac60-57b3-4f1d-b4ef-056f39f8f1d7 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:48.359151 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 186/382] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 1 13:55:48 2022] GET /identity/v3/role_assignments?group.id=f73bbef49520470cb75b98a774a1bcfe&scope.project.id=b30b7ebe5b2d4e7c8f161e5b78a7a0df&role.id=58f91e3a13664dabb18c6227a1658fa6 => generated 260 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:48.403803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460acfc-5ff3-44f8-ba92-a09943294209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:48.404779 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460acfc-5ff3-44f8-ba92-a09943294209 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:48.405248 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460acfc-5ff3-44f8-ba92-a09943294209 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:48.405684 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460acfc-5ff3-44f8-ba92-a09943294209 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:48.419185 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f460acfc-5ff3-44f8-ba92-a09943294209 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:48.424816 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-f460acfc-5ff3-44f8-ba92-a09943294209 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:55:48.424816 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:48.424816 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:48.424816 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:48.424816 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:48.424816 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:48.424816 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:48.424816 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:48.424816 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:48.424816 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:48.424816 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:48.424816 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:48.424816 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:48.424816 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:48.424816 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:48.424816 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:48.425960 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:48.425960 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:48.425960 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:48.425960 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:48.425960 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:48.425960 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:48.425960 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:48.425960 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:48.425960 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:48.425960 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:48.425960 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:48.425960 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:48.425960 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:48.425960 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:48.425960 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:48.425960 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:48.426916 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:48.426916 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:48.426916 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:48.426916 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:48.426916 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:48.426916 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:48.426916 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:48.426916 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:48.426916 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:48.426916 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:48.426916 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:55:48.426916 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:55:48.434487 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 197/383] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:55:48.436159 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fa1e89-8e54-4af8-ba02-5bc7284bcdb4 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:48.498698 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:48.565576 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fa1e89-8e54-4af8-ba02-5bc7284bcdb4 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:48.607016 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af330005-443c-4852-968c-df9809fe75be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:48.608017 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af330005-443c-4852-968c-df9809fe75be None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:48.608594 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af330005-443c-4852-968c-df9809fe75be None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:48.609145 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af330005-443c-4852-968c-df9809fe75be None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:48.610109 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af330005-443c-4852-968c-df9809fe75be None admin] RBAC: Adding query filter params (name=member) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:48.610615 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af330005-443c-4852-968c-df9809fe75be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:48.612230 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af330005-443c-4852-968c-df9809fe75be None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:48.623306 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 187/384] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 13:55:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:48.628750 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460acfc-5ff3-44f8-ba92-a09943294209 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:48.749528 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460acfc-5ff3-44f8-ba92-a09943294209 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:48.790147 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34767af2-9b84-42c5-8a04-de312b730e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:48.790810 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34767af2-9b84-42c5-8a04-de312b730e83 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:48.791092 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34767af2-9b84-42c5-8a04-de312b730e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:48.791895 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34767af2-9b84-42c5-8a04-de312b730e83 None admin] PATH_INFO: `/v3/groups/158f3c1b59904f1f93ecd8987d832191` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:48.793118 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34767af2-9b84-42c5-8a04-de312b730e83 None admin] RBAC: Authorizing `identity:get_group(group_id=158f3c1b59904f1f93ecd8987d832191)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:48.794557 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34767af2-9b84-42c5-8a04-de312b730e83 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:48.796311 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 198/385] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 13:55:48 2022] GET /identity/v3/groups/158f3c1b59904f1f93ecd8987d832191 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:48.801841 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af330005-443c-4852-968c-df9809fe75be None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:48.949809 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af330005-443c-4852-968c-df9809fe75be None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:48.964730 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:48.998063 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356215ea-e914-4483-acfc-4a099a410b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:48.999082 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356215ea-e914-4483-acfc-4a099a410b94 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:48.999591 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356215ea-e914-4483-acfc-4a099a410b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:49.000086 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356215ea-e914-4483-acfc-4a099a410b94 None admin] PATH_INFO: `/v3/projects/516ee04b71684e5885fa9a08b821fb87` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:49.005207 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356215ea-e914-4483-acfc-4a099a410b94 None admin] RBAC: Authorizing `identity:get_project(project_id=516ee04b71684e5885fa9a08b821fb87)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:49.006887 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356215ea-e914-4483-acfc-4a099a410b94 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:49.009144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 188/386] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:48 2022] GET /identity/v3/projects/516ee04b71684e5885fa9a08b821fb87 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:49.014956 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34767af2-9b84-42c5-8a04-de312b730e83 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:49.056108 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:49.163017 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34767af2-9b84-42c5-8a04-de312b730e83 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:49.209368 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73557844-463d-48ed-a90a-015d73acdf37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:49.209983 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73557844-463d-48ed-a90a-015d73acdf37 None admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:49.210270 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73557844-463d-48ed-a90a-015d73acdf37 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:49.210551 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73557844-463d-48ed-a90a-015d73acdf37 None admin] PATH_INFO: `/v3/projects/516ee04b71684e5885fa9a08b821fb87/groups/158f3c1b59904f1f93ecd8987d832191/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:49.212530 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73557844-463d-48ed-a90a-015d73acdf37 None admin] RBAC: Authorizing `identity:create_grant(project_id=516ee04b71684e5885fa9a08b821fb87, group_id=158f3c1b59904f1f93ecd8987d832191, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:49.214514 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73557844-463d-48ed-a90a-015d73acdf37 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:49.237420 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 199/387] 10.222.0.48 () {58 vars in 1442 bytes} [Mon Aug 1 13:55:49 2022] PUT /identity/v3/projects/516ee04b71684e5885fa9a08b821fb87/groups/158f3c1b59904f1f93ecd8987d832191/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:55:49.260151 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10ffb3-2fb2-41e2-bf89-5eb07a108a2f None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:49.260817 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10ffb3-2fb2-41e2-bf89-5eb07a108a2f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:49.261315 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10ffb3-2fb2-41e2-bf89-5eb07a108a2f None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:49.263235 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 189/388] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:49.268634 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d2e6c-e590-4387-a99b-0ef812b9d86f None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:49.268923 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d2e6c-e590-4387-a99b-0ef812b9d86f None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:49.269212 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d2e6c-e590-4387-a99b-0ef812b9d86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:49.294004 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-058d2e6c-e590-4387-a99b-0ef812b9d86f None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:49.295228 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-058d2e6c-e590-4387-a99b-0ef812b9d86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:49.381128 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 200/389] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:49.413059 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b10ffb3-2fb2-41e2-bf89-5eb07a108a2f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:49.540959 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b10ffb3-2fb2-41e2-bf89-5eb07a108a2f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:49.579664 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b160e-2892-4756-b5a8-177154343e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:49.580332 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b160e-2892-4756-b5a8-177154343e73 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:49.580652 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b160e-2892-4756-b5a8-177154343e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:49.580935 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b160e-2892-4756-b5a8-177154343e73 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:49.592597 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b160e-2892-4756-b5a8-177154343e73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:49.598116 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-122b160e-2892-4756-b5a8-177154343e73 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:55:49.598116 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:49.598116 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:49.598116 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:49.598116 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:49.598116 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:49.598116 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:49.598116 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:49.598116 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:49.598116 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:49.598116 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:49.598116 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:49.598116 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:49.598116 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:49.598116 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:49.598116 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:49.599008 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:49.599008 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:49.599008 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:49.599008 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:49.599008 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:49.599008 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:49.599008 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:49.599008 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:49.599008 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:49.599008 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:49.599008 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:49.599008 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:49.599008 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:49.599008 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:49.599008 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:49.599008 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:49.599813 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:49.599813 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:49.599813 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:49.599813 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:49.599813 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:49.599813 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:49.599813 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:49.599813 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:49.599813 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:49.599813 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:49.599813 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:55:49.599813 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:55:49.599813 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 190/390] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 13:55:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:49.605065 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058d2e6c-e590-4387-a99b-0ef812b9d86f None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:49.630124 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:49.768963 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058d2e6c-e590-4387-a99b-0ef812b9d86f None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:49.821107 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa2e10-bb3a-4ca0-a8b1-4a74aa259dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:49.821745 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa2e10-bb3a-4ca0-a8b1-4a74aa259dfa None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:49.822030 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa2e10-bb3a-4ca0-a8b1-4a74aa259dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:49.822376 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa2e10-bb3a-4ca0-a8b1-4a74aa259dfa None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:49.823717 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa2e10-bb3a-4ca0-a8b1-4a74aa259dfa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:49.824029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa2e10-bb3a-4ca0-a8b1-4a74aa259dfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:49.825434 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa2e10-bb3a-4ca0-a8b1-4a74aa259dfa None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:49.840163 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 201/391] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:55:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:49.845868 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b160e-2892-4756-b5a8-177154343e73 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:49.906485 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:49.981377 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b160e-2892-4756-b5a8-177154343e73 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:50.025202 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283b5e8-f0fb-4faf-a21a-8a3867604be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:50.026386 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283b5e8-f0fb-4faf-a21a-8a3867604be7 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:50.026678 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283b5e8-f0fb-4faf-a21a-8a3867604be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:50.026958 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283b5e8-f0fb-4faf-a21a-8a3867604be7 None admin] PATH_INFO: `/v3/groups/158f3c1b59904f1f93ecd8987d832191` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:50.028172 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283b5e8-f0fb-4faf-a21a-8a3867604be7 None admin] RBAC: Authorizing `identity:get_group(group_id=158f3c1b59904f1f93ecd8987d832191)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:50.029637 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283b5e8-f0fb-4faf-a21a-8a3867604be7 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:50.035712 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 191/392] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 13:55:49 2022] GET /identity/v3/groups/158f3c1b59904f1f93ecd8987d832191 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:50.037558 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa2e10-bb3a-4ca0-a8b1-4a74aa259dfa None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:50.159372 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:50.204009 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa2e10-bb3a-4ca0-a8b1-4a74aa259dfa None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:50.231752 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a63ad5e-90a6-495e-bffc-1028596dcfc6 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:50.232056 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a63ad5e-90a6-495e-bffc-1028596dcfc6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:50.232328 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a63ad5e-90a6-495e-bffc-1028596dcfc6 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:50.234963 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 192/393] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:50.239063 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724ce72-3d8e-4416-b613-5824bb08753d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:50.239355 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724ce72-3d8e-4416-b613-5824bb08753d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:50.239634 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724ce72-3d8e-4416-b613-5824bb08753d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:50.274059 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48aaeb-bdc3-4a40-ad4d-516dae120d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:50.275125 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48aaeb-bdc3-4a40-ad4d-516dae120d6d None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:50.276224 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48aaeb-bdc3-4a40-ad4d-516dae120d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:50.276900 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48aaeb-bdc3-4a40-ad4d-516dae120d6d None admin] PATH_INFO: `/v3/projects/516ee04b71684e5885fa9a08b821fb87` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:50.278971 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f48aaeb-bdc3-4a40-ad4d-516dae120d6d None admin] RBAC: Authorizing `identity:get_project(project_id=516ee04b71684e5885fa9a08b821fb87)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:50.281412 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-6724ce72-3d8e-4416-b613-5824bb08753d None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:50.281627 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f48aaeb-bdc3-4a40-ad4d-516dae120d6d None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:50.282745 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6724ce72-3d8e-4416-b613-5824bb08753d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:50.284974 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 202/394] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:50 2022] GET /identity/v3/projects/516ee04b71684e5885fa9a08b821fb87 => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:50.293709 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48aaeb-bdc3-4a40-ad4d-516dae120d6d None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:50.386510 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 193/395] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:50.409067 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6724ce72-3d8e-4416-b613-5824bb08753d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:50.445037 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:50.490891 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48aaeb-bdc3-4a40-ad4d-516dae120d6d None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:50.521098 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6724ce72-3d8e-4416-b613-5824bb08753d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:50.558071 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cde45b-5df2-409f-ba1b-07b19f0ae72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:50.558812 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cde45b-5df2-409f-ba1b-07b19f0ae72f None admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:50.559107 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cde45b-5df2-409f-ba1b-07b19f0ae72f None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:50.559390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cde45b-5df2-409f-ba1b-07b19f0ae72f None admin] PATH_INFO: `/v3/projects/516ee04b71684e5885fa9a08b821fb87/groups/158f3c1b59904f1f93ecd8987d832191/roles/47e170213d8049a28f8b8a0d736fdf6b` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:50.566100 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cde45b-5df2-409f-ba1b-07b19f0ae72f None admin] RBAC: Authorizing `identity:create_grant(project_id=516ee04b71684e5885fa9a08b821fb87, group_id=158f3c1b59904f1f93ecd8987d832191, role_id=47e170213d8049a28f8b8a0d736fdf6b)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:50.567564 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cde45b-5df2-409f-ba1b-07b19f0ae72f None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:50.586272 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:50.587147 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f6cdc4-2630-4c28-935a-e7d1b86f9cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:50.587814 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6cdc4-2630-4c28-935a-e7d1b86f9cc1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:50.588128 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6cdc4-2630-4c28-935a-e7d1b86f9cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:50.588407 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6cdc4-2630-4c28-935a-e7d1b86f9cc1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:50.591562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 203/396] 10.222.0.48 () {58 vars in 1442 bytes} [Mon Aug 1 13:55:50 2022] PUT /identity/v3/projects/516ee04b71684e5885fa9a08b821fb87/groups/158f3c1b59904f1f93ecd8987d832191/roles/47e170213d8049a28f8b8a0d736fdf6b => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 13:55:50.625413 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f6cdc4-2630-4c28-935a-e7d1b86f9cc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:50.630579 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-82f6cdc4-2630-4c28-935a-e7d1b86f9cc1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:55:50.630579 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:50.630579 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:50.630579 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:50.630579 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:50.630579 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:50.630579 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:50.630579 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:50.630579 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:50.630579 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:50.630579 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:50.630579 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:50.630579 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:50.630579 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:50.630579 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:50.630579 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:50.631689 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:50.631689 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:50.631689 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:50.631689 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:50.631689 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:50.631689 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:50.631689 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:50.631689 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:50.631689 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:50.631689 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:50.631689 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:50.631689 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:50.631689 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:50.631689 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:50.631689 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:50.631689 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:50.632770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:50.632770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:50.632770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:50.632770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:50.632770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:50.632770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:50.632770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:50.632770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:50.632770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:50.632770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:50.632770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:55:50.632770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:55:50.632770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 194/397] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:55:50.639647 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cde45b-5df2-409f-ba1b-07b19f0ae72f None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:50.823431 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cde45b-5df2-409f-ba1b-07b19f0ae72f None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:50.875841 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17256c56-3e12-45a5-9d5b-d747917ab494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:50.876910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17256c56-3e12-45a5-9d5b-d747917ab494 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:50.877393 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17256c56-3e12-45a5-9d5b-d747917ab494 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:50.877864 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17256c56-3e12-45a5-9d5b-d747917ab494 None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:50.879157 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17256c56-3e12-45a5-9d5b-d747917ab494 None admin] RBAC: Adding query filter params (name=member) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:50.879689 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17256c56-3e12-45a5-9d5b-d747917ab494 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:50.882193 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17256c56-3e12-45a5-9d5b-d747917ab494 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:50.897779 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6f3c2-e98f-42ff-a580-dea617239ca8 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:50.898097 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6f3c2-e98f-42ff-a580-dea617239ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:50.898342 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6f3c2-e98f-42ff-a580-dea617239ca8 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:50.901536 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 195/398] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:50.903202 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 204/399] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 13:55:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:50.906857 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20317da-26ed-4a71-8f16-58431a173ca8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:50.907140 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20317da-26ed-4a71-8f16-58431a173ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:50.907656 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20317da-26ed-4a71-8f16-58431a173ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:50.909281 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17256c56-3e12-45a5-9d5b-d747917ab494 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:50.928786 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d20317da-26ed-4a71-8f16-58431a173ca8 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:50.930004 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d20317da-26ed-4a71-8f16-58431a173ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:51.014930 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:51.062798 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17256c56-3e12-45a5-9d5b-d747917ab494 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:51.067104 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 196/400] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:55:51.105650 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20317da-26ed-4a71-8f16-58431a173ca8 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:51.106189 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511fecfa-b5d6-44be-9339-15b338adef7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:51.107056 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511fecfa-b5d6-44be-9339-15b338adef7b None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:51.107799 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511fecfa-b5d6-44be-9339-15b338adef7b None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:51.108256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511fecfa-b5d6-44be-9339-15b338adef7b None admin] PATH_INFO: `/v3/groups/158f3c1b59904f1f93ecd8987d832191` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:51.109657 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511fecfa-b5d6-44be-9339-15b338adef7b None admin] RBAC: Authorizing `identity:get_group(group_id=158f3c1b59904f1f93ecd8987d832191)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:51.111280 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511fecfa-b5d6-44be-9339-15b338adef7b None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:51.113434 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 205/401] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 13:55:50 2022] GET /identity/v3/groups/158f3c1b59904f1f93ecd8987d832191 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:51.120509 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511fecfa-b5d6-44be-9339-15b338adef7b None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:51.159098 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:51.250049 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20317da-26ed-4a71-8f16-58431a173ca8 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:51.296628 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f0e39-3d95-464e-b28b-be6f45c8a41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:51.296952 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511fecfa-b5d6-44be-9339-15b338adef7b None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:51.297830 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f0e39-3d95-464e-b28b-be6f45c8a41a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:51.298297 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f0e39-3d95-464e-b28b-be6f45c8a41a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:51.298883 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f0e39-3d95-464e-b28b-be6f45c8a41a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:51.313864 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576f0e39-3d95-464e-b28b-be6f45c8a41a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:51.319477 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-576f0e39-3d95-464e-b28b-be6f45c8a41a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:55:51.319477 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:51.319477 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:51.319477 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:51.319477 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:51.319477 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:51.319477 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:51.319477 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:51.319477 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:51.319477 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:51.319477 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:51.319477 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:51.319477 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:51.319477 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:51.319477 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:51.319477 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:51.320354 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:51.320354 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:51.320354 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:51.320354 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:51.320354 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:51.320354 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:51.320354 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:51.320354 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:51.320354 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:51.320354 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:51.320354 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:51.320354 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:51.320354 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:51.320354 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:51.320354 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:51.320354 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:51.321260 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:51.321260 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:51.321260 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:51.321260 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:51.321260 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:51.321260 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:51.321260 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:51.321260 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:51.321260 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:51.321260 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:51.321260 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:55:51.321260 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:55:51.321260 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 197/402] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 13:55:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:51.322748 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05260f3-56a6-4aae-bb51-58ae8c5b978e None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:51.323104 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05260f3-56a6-4aae-bb51-58ae8c5b978e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:51.323454 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05260f3-56a6-4aae-bb51-58ae8c5b978e None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:51.329044 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 198/403] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:51.330016 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05260f3-56a6-4aae-bb51-58ae8c5b978e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:51.339488 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:51.347530 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70721c93-6e88-4c16-8353-00e8417d8f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:51.348567 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70721c93-6e88-4c16-8353-00e8417d8f58 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:51.349460 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70721c93-6e88-4c16-8353-00e8417d8f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:51.350070 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70721c93-6e88-4c16-8353-00e8417d8f58 None admin] PATH_INFO: `/v3/projects/05e772d8318b47469008ecb063004d71` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:51.351562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70721c93-6e88-4c16-8353-00e8417d8f58 None admin] RBAC: Authorizing `identity:get_project(project_id=05e772d8318b47469008ecb063004d71)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:51.354668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70721c93-6e88-4c16-8353-00e8417d8f58 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:51.357639 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 206/404] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:51 2022] GET /identity/v3/projects/05e772d8318b47469008ecb063004d71 => generated 302 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:51.359070 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40715b50-c56c-45e0-a82e-1ba7be8edaa8 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:51.359524 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40715b50-c56c-45e0-a82e-1ba7be8edaa8 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:51.359964 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40715b50-c56c-45e0-a82e-1ba7be8edaa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:51.383281 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-40715b50-c56c-45e0-a82e-1ba7be8edaa8 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:51.384833 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-40715b50-c56c-45e0-a82e-1ba7be8edaa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:51.438358 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:51.458254 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05260f3-56a6-4aae-bb51-58ae8c5b978e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:51.479124 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 207/405] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:51.480235 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40715b50-c56c-45e0-a82e-1ba7be8edaa8 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:51.517167 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075d744-8eb2-4786-861c-b6cc4db477b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:51.517957 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075d744-8eb2-4786-861c-b6cc4db477b2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:51.518357 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075d744-8eb2-4786-861c-b6cc4db477b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:51.518713 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075d744-8eb2-4786-861c-b6cc4db477b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:51.519656 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2075d744-8eb2-4786-861c-b6cc4db477b2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:51.520051 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2075d744-8eb2-4786-861c-b6cc4db477b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:51.521948 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2075d744-8eb2-4786-861c-b6cc4db477b2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:51.536580 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 199/406] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:55:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:51.537779 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075d744-8eb2-4786-861c-b6cc4db477b2 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:51.607731 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40715b50-c56c-45e0-a82e-1ba7be8edaa8 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:51.645308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:51.656855 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee161168-debc-4e91-bad1-74d1b5e35cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:51.657759 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee161168-debc-4e91-bad1-74d1b5e35cf5 None admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:51.658008 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075d744-8eb2-4786-861c-b6cc4db477b2 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:51.658646 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee161168-debc-4e91-bad1-74d1b5e35cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:51.659070 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee161168-debc-4e91-bad1-74d1b5e35cf5 None admin] PATH_INFO: `/v3/projects/05e772d8318b47469008ecb063004d71/groups/158f3c1b59904f1f93ecd8987d832191/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:51.662410 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee161168-debc-4e91-bad1-74d1b5e35cf5 None admin] RBAC: Authorizing `identity:create_grant(project_id=05e772d8318b47469008ecb063004d71, group_id=158f3c1b59904f1f93ecd8987d832191, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:51.664023 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee161168-debc-4e91-bad1-74d1b5e35cf5 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:51.681869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 208/407] 10.222.0.48 () {58 vars in 1442 bytes} [Mon Aug 1 13:55:51 2022] PUT /identity/v3/projects/05e772d8318b47469008ecb063004d71/groups/158f3c1b59904f1f93ecd8987d832191/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:55:51.682920 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee161168-debc-4e91-bad1-74d1b5e35cf5 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:51.726998 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35886323-8dc9-4f38-ae93-e351175eb58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:51.728092 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35886323-8dc9-4f38-ae93-e351175eb58d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:51.728652 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35886323-8dc9-4f38-ae93-e351175eb58d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:51.729134 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35886323-8dc9-4f38-ae93-e351175eb58d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:51.749076 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35886323-8dc9-4f38-ae93-e351175eb58d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:51.755828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-35886323-8dc9-4f38-ae93-e351175eb58d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:55:51.755828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:51.755828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:51.755828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:51.755828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:51.755828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:51.755828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:51.755828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:51.755828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:51.755828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:51.755828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:51.755828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:51.755828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:51.755828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:51.755828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:51.755828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:51.756828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:51.756828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:51.756828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:51.756828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:51.756828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:51.756828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:51.756828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:51.756828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:51.756828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:51.756828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:51.756828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:51.756828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:51.756828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:51.756828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:51.756828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:51.756828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:51.757876 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:51.757876 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:51.757876 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:51.757876 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:51.757876 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:51.757876 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:51.757876 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:51.757876 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:51.757876 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:51.757876 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:51.757876 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:55:51.757876 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:55:51.766359 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 200/408] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 13:55:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:51.773741 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35886323-8dc9-4f38-ae93-e351175eb58d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:51.793413 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:51.908111 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee161168-debc-4e91-bad1-74d1b5e35cf5 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:51.950567 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5433afdc-862c-4966-b4d3-cd0e2ef7dcc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:51.951430 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433afdc-862c-4966-b4d3-cd0e2ef7dcc3 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:51.951860 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433afdc-862c-4966-b4d3-cd0e2ef7dcc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:51.952237 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433afdc-862c-4966-b4d3-cd0e2ef7dcc3 None admin] PATH_INFO: `/v3/groups/158f3c1b59904f1f93ecd8987d832191` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:51.953595 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5433afdc-862c-4966-b4d3-cd0e2ef7dcc3 None admin] RBAC: Authorizing `identity:get_group(group_id=158f3c1b59904f1f93ecd8987d832191)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:51.955870 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5433afdc-862c-4966-b4d3-cd0e2ef7dcc3 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:51.958755 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 209/409] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 13:55:51 2022] GET /identity/v3/groups/158f3c1b59904f1f93ecd8987d832191 => generated 226 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:51.969852 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5433afdc-862c-4966-b4d3-cd0e2ef7dcc3 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:52.006289 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35886323-8dc9-4f38-ae93-e351175eb58d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:52.047679 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:52.053646 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4abef61-c89d-4be0-b5ad-e5703bb5a2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:52.054292 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4abef61-c89d-4be0-b5ad-e5703bb5a2ce None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:52.054580 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4abef61-c89d-4be0-b5ad-e5703bb5a2ce None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:52.054868 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4abef61-c89d-4be0-b5ad-e5703bb5a2ce None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:52.055858 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4abef61-c89d-4be0-b5ad-e5703bb5a2ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:52.056168 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4abef61-c89d-4be0-b5ad-e5703bb5a2ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:52.058770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4abef61-c89d-4be0-b5ad-e5703bb5a2ce None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:52.070136 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 201/410] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:52.071411 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b78ab-8e83-4dda-b1f2-dbc78ba0d7ae None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:52.071703 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b78ab-8e83-4dda-b1f2-dbc78ba0d7ae None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:52.071984 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b78ab-8e83-4dda-b1f2-dbc78ba0d7ae None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:52.074087 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 202/411] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:52.077787 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b78ab-8e83-4dda-b1f2-dbc78ba0d7ae None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:52.112295 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5433afdc-862c-4966-b4d3-cd0e2ef7dcc3 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:52.158222 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:52.158394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6015d53b-cad3-471e-adaa-f1cfb1eaedc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:52.159359 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015d53b-cad3-471e-adaa-f1cfb1eaedc7 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:52.159690 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015d53b-cad3-471e-adaa-f1cfb1eaedc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:52.160099 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015d53b-cad3-471e-adaa-f1cfb1eaedc7 None admin] PATH_INFO: `/v3/projects/05e772d8318b47469008ecb063004d71` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:52.162942 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6015d53b-cad3-471e-adaa-f1cfb1eaedc7 None admin] RBAC: Authorizing `identity:get_project(project_id=05e772d8318b47469008ecb063004d71)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:52.164409 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6015d53b-cad3-471e-adaa-f1cfb1eaedc7 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:52.166477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 210/412] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:51 2022] GET /identity/v3/projects/05e772d8318b47469008ecb063004d71 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:52.167727 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15722b45-2f67-4c39-9794-5a64a64a042e None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:52.168017 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15722b45-2f67-4c39-9794-5a64a64a042e None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:52.168293 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15722b45-2f67-4c39-9794-5a64a64a042e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:52.191939 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-15722b45-2f67-4c39-9794-5a64a64a042e None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:52.193844 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-15722b45-2f67-4c39-9794-5a64a64a042e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:52.200461 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b78ab-8e83-4dda-b1f2-dbc78ba0d7ae None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:52.243675 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0bb06a-27d5-4182-850f-4e2aaf295249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:52.244992 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bb06a-27d5-4182-850f-4e2aaf295249 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:52.245303 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bb06a-27d5-4182-850f-4e2aaf295249 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:52.245582 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bb06a-27d5-4182-850f-4e2aaf295249 None admin] PATH_INFO: `/v3/groups/f73bbef49520470cb75b98a774a1bcfe` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:52.246827 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0bb06a-27d5-4182-850f-4e2aaf295249 None admin] RBAC: Authorizing `identity:get_group(group_id=f73bbef49520470cb75b98a774a1bcfe)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:52.248302 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0bb06a-27d5-4182-850f-4e2aaf295249 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:52.250388 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 203/413] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 13:55:52 2022] GET /identity/v3/groups/f73bbef49520470cb75b98a774a1bcfe => generated 229 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:52.251154 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0bb06a-27d5-4182-850f-4e2aaf295249 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:52.277488 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 211/414] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:52.278543 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15722b45-2f67-4c39-9794-5a64a64a042e None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:52.348848 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:52.362610 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0bb06a-27d5-4182-850f-4e2aaf295249 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:52.400017 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15722b45-2f67-4c39-9794-5a64a64a042e None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:52.403710 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b232b7-fb0c-4992-84eb-f54ade06eea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:52.404782 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b232b7-fb0c-4992-84eb-f54ade06eea2 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:52.405250 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b232b7-fb0c-4992-84eb-f54ade06eea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:52.405716 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b232b7-fb0c-4992-84eb-f54ade06eea2 None admin] PATH_INFO: `/v3/projects/05e772d8318b47469008ecb063004d71/groups/158f3c1b59904f1f93ecd8987d832191/roles/47e170213d8049a28f8b8a0d736fdf6b` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:52.408360 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b232b7-fb0c-4992-84eb-f54ade06eea2 None admin] RBAC: Authorizing `identity:create_grant(project_id=05e772d8318b47469008ecb063004d71, group_id=158f3c1b59904f1f93ecd8987d832191, role_id=47e170213d8049a28f8b8a0d736fdf6b)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:52.410225 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b232b7-fb0c-4992-84eb-f54ade06eea2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:52.421665 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:52.427839 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 204/415] 10.222.0.48 () {58 vars in 1442 bytes} [Mon Aug 1 13:55:52 2022] PUT /identity/v3/projects/05e772d8318b47469008ecb063004d71/groups/158f3c1b59904f1f93ecd8987d832191/roles/47e170213d8049a28f8b8a0d736fdf6b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:55:52.429332 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b232b7-fb0c-4992-84eb-f54ade06eea2 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:52.443909 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7018ec2-53f0-4863-9af2-5a812b8add2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:52.444319 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7018ec2-53f0-4863-9af2-5a812b8add2a None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:52.444637 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7018ec2-53f0-4863-9af2-5a812b8add2a None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:52.444940 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7018ec2-53f0-4863-9af2-5a812b8add2a None admin] PATH_INFO: `/v3/projects/b30b7ebe5b2d4e7c8f161e5b78a7a0df` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:52.446282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7018ec2-53f0-4863-9af2-5a812b8add2a None admin] RBAC: Authorizing `identity:get_project(project_id=b30b7ebe5b2d4e7c8f161e5b78a7a0df)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:52.448513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7018ec2-53f0-4863-9af2-5a812b8add2a None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:52.450756 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 212/416] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:52 2022] GET /identity/v3/projects/b30b7ebe5b2d4e7c8f161e5b78a7a0df => generated 344 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:52.458758 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7018ec2-53f0-4863-9af2-5a812b8add2a None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:52.548753 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b232b7-fb0c-4992-84eb-f54ade06eea2 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:52.587448 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65356fbe-b331-4bfe-84bd-cf110d926e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:52.589116 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65356fbe-b331-4bfe-84bd-cf110d926e72 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:52.589614 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65356fbe-b331-4bfe-84bd-cf110d926e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:52.590083 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65356fbe-b331-4bfe-84bd-cf110d926e72 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:52.602035 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65356fbe-b331-4bfe-84bd-cf110d926e72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:52.608578 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-65356fbe-b331-4bfe-84bd-cf110d926e72 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:55:52.608578 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:52.608578 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:52.608578 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:52.608578 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:52.608578 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:52.608578 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:52.608578 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:52.608578 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:52.608578 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:52.608578 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:52.608578 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:52.608578 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:52.608578 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:52.608578 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:52.608578 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:52.609468 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:52.609468 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:52.609468 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:52.609468 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:52.609468 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:52.609468 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:52.609468 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:52.609468 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:52.609468 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:52.609468 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:52.609468 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:52.609468 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:52.609468 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:52.609468 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:52.609468 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:52.609468 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:52.610214 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:52.610214 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:52.610214 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:52.610214 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:52.610214 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:52.610214 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:52.610214 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:52.610214 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:52.610214 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:52.610214 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:52.610214 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:55:52.610214 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:55:52.613035 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 205/417] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:52.618251 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65356fbe-b331-4bfe-84bd-cf110d926e72 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:52.641979 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7018ec2-53f0-4863-9af2-5a812b8add2a None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:52.686722 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:52.718193 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:52.721828 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01883e3-f81d-44fe-97d5-a356efa991c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:52.722998 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01883e3-f81d-44fe-97d5-a356efa991c5 None admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:52.724713 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01883e3-f81d-44fe-97d5-a356efa991c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:52.725232 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01883e3-f81d-44fe-97d5-a356efa991c5 None admin] PATH_INFO: `/v3/projects/b30b7ebe5b2d4e7c8f161e5b78a7a0df/groups/f73bbef49520470cb75b98a774a1bcfe/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:52.732984 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01883e3-f81d-44fe-97d5-a356efa991c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=b30b7ebe5b2d4e7c8f161e5b78a7a0df, group_id=f73bbef49520470cb75b98a774a1bcfe, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:52.734842 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01883e3-f81d-44fe-97d5-a356efa991c5 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:52.756737 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 213/418] 10.222.0.48 () {58 vars in 1442 bytes} [Mon Aug 1 13:55:52 2022] PUT /identity/v3/projects/b30b7ebe5b2d4e7c8f161e5b78a7a0df/groups/f73bbef49520470cb75b98a774a1bcfe/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:55:52.783693 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65356fbe-b331-4bfe-84bd-cf110d926e72 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:52.834384 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc906e-0ce7-4ed1-a0d5-df0331e8f4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:52.835042 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc906e-0ce7-4ed1-a0d5-df0331e8f4b0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:52.835328 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc906e-0ce7-4ed1-a0d5-df0331e8f4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:52.835609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc906e-0ce7-4ed1-a0d5-df0331e8f4b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:52.836430 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cc906e-0ce7-4ed1-a0d5-df0331e8f4b0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:52.836779 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cc906e-0ce7-4ed1-a0d5-df0331e8f4b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:52.838398 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cc906e-0ce7-4ed1-a0d5-df0331e8f4b0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:52.849294 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 206/419] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 13:55:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:52.856513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01883e3-f81d-44fe-97d5-a356efa991c5 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:52.933122 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df0095-e4d7-4189-b7e5-e1012a65debc None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:52.933445 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df0095-e4d7-4189-b7e5-e1012a65debc None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:52.933688 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df0095-e4d7-4189-b7e5-e1012a65debc None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:52.935063 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 207/420] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:55:52.940768 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50e838-8734-419d-94ec-3f4c5bb76315 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:52.941320 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50e838-8734-419d-94ec-3f4c5bb76315 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:52.941788 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50e838-8734-419d-94ec-3f4c5bb76315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:52.963005 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-7a50e838-8734-419d-94ec-3f4c5bb76315 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:52.964632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7a50e838-8734-419d-94ec-3f4c5bb76315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:53.002844 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01883e3-f81d-44fe-97d5-a356efa991c5 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:53.054619 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb67cc-e315-4d9b-bcb4-9eacfa66de47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:53.057943 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb67cc-e315-4d9b-bcb4-9eacfa66de47 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:53.057943 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb67cc-e315-4d9b-bcb4-9eacfa66de47 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:53.057943 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb67cc-e315-4d9b-bcb4-9eacfa66de47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:53.090200 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 208/421] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:55:53.118191 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:53.127670 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50e838-8734-419d-94ec-3f4c5bb76315 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:53.133282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffb67cc-e315-4d9b-bcb4-9eacfa66de47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:53.134815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffb67cc-e315-4d9b-bcb4-9eacfa66de47 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:53.274652 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 214/422] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:55:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:55:53.277546 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb67cc-e315-4d9b-bcb4-9eacfa66de47 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:53.332583 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:53.399046 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:53.422806 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb67cc-e315-4d9b-bcb4-9eacfa66de47 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:53.460500 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819058f-4794-4c4b-aec8-d9a39c467ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:53.461865 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819058f-4794-4c4b-aec8-d9a39c467ddd None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:53.462144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819058f-4794-4c4b-aec8-d9a39c467ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:53.462419 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819058f-4794-4c4b-aec8-d9a39c467ddd None admin] PATH_INFO: `/v3/projects/516ee04b71684e5885fa9a08b821fb87` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:53.463674 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8819058f-4794-4c4b-aec8-d9a39c467ddd None admin] RBAC: Authorizing `identity:get_project(project_id=516ee04b71684e5885fa9a08b821fb87)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:53.465541 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8819058f-4794-4c4b-aec8-d9a39c467ddd None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:53.472544 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 215/423] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:53 2022] GET /identity/v3/projects/516ee04b71684e5885fa9a08b821fb87 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:53.474142 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819058f-4794-4c4b-aec8-d9a39c467ddd None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:53.529789 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50e838-8734-419d-94ec-3f4c5bb76315 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:53.615714 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819058f-4794-4c4b-aec8-d9a39c467ddd None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:53.617347 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4b7ce-36bf-421f-97e6-c0c03e0fbd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:53.618244 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4b7ce-36bf-421f-97e6-c0c03e0fbd53 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:53.618720 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4b7ce-36bf-421f-97e6-c0c03e0fbd53 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:53.619788 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4b7ce-36bf-421f-97e6-c0c03e0fbd53 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:53.659733 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a4b7ce-36bf-421f-97e6-c0c03e0fbd53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:53.662784 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1836aae-ebbb-497e-81b6-00e2c0fb1503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:53.663492 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1836aae-ebbb-497e-81b6-00e2c0fb1503 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:53.663783 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1836aae-ebbb-497e-81b6-00e2c0fb1503 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:53.664064 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1836aae-ebbb-497e-81b6-00e2c0fb1503 None admin] PATH_INFO: `/v3/groups/158f3c1b59904f1f93ecd8987d832191` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:53.666486 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-39a4b7ce-36bf-421f-97e6-c0c03e0fbd53 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:55:53.666486 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:53.666486 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:53.666486 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:53.666486 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:53.666486 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:53.666486 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:53.666486 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:53.666486 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:53.666486 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:53.666486 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:53.666486 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:53.666486 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:53.666486 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:53.666486 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:53.666486 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:53.667333 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:53.667333 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:53.667333 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:53.667333 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:53.667333 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:53.667333 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:53.667333 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:53.667333 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:53.667333 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:53.667333 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:53.667333 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:53.667333 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:53.667333 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:53.667333 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:53.667333 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:53.667333 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:53.668108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:53.668108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:53.668108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:53.668108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:53.668108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:53.668108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:53.668108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:53.668108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:53.668108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:53.668108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:53.668108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:55:53.668108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:55:53.677515 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1836aae-ebbb-497e-81b6-00e2c0fb1503 None admin] RBAC: Authorizing `identity:get_group(group_id=158f3c1b59904f1f93ecd8987d832191)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:53.677515 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1836aae-ebbb-497e-81b6-00e2c0fb1503 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:53.677515 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 216/424] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 13:55:53 2022] GET /identity/v3/groups/158f3c1b59904f1f93ecd8987d832191 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:53.677515 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1836aae-ebbb-497e-81b6-00e2c0fb1503 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:53.679284 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 209/425] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 13:55:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 552 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:55:53.689666 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4b7ce-36bf-421f-97e6-c0c03e0fbd53 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:53.703526 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:53.824864 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1836aae-ebbb-497e-81b6-00e2c0fb1503 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:53.862567 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed31bf-81a7-498d-b313-2b82bd7bb3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:53.863228 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed31bf-81a7-498d-b313-2b82bd7bb3c9 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:53.863843 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed31bf-81a7-498d-b313-2b82bd7bb3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:53.863953 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed31bf-81a7-498d-b313-2b82bd7bb3c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:53.864858 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed31bf-81a7-498d-b313-2b82bd7bb3c9 None admin] RBAC: Adding query filter params (group.id=158f3c1b59904f1f93ecd8987d832191, role.id=926b9f36c24847c582c84af904250abc, scope.project.id=516ee04b71684e5885fa9a08b821fb87) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:53.865174 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed31bf-81a7-498d-b313-2b82bd7bb3c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:53.866616 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed31bf-81a7-498d-b313-2b82bd7bb3c9 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:53.873807 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 217/426] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 1 13:55:53 2022] GET /identity/v3/role_assignments?group.id=158f3c1b59904f1f93ecd8987d832191&scope.project.id=516ee04b71684e5885fa9a08b821fb87&role.id=926b9f36c24847c582c84af904250abc => generated 611 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:54.032854 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4b7ce-36bf-421f-97e6-c0c03e0fbd53 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:54.044601 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:54.071291 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9753fed-6dcb-463e-81e2-027bf742f6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:54.071812 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9753fed-6dcb-463e-81e2-027bf742f6c9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:54.072635 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9753fed-6dcb-463e-81e2-027bf742f6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:54.073114 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9753fed-6dcb-463e-81e2-027bf742f6c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:54.074442 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9753fed-6dcb-463e-81e2-027bf742f6c9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:54.074865 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9753fed-6dcb-463e-81e2-027bf742f6c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:54.076324 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9753fed-6dcb-463e-81e2-027bf742f6c9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:54.085469 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 210/427] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:55:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:54.091607 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed31bf-81a7-498d-b313-2b82bd7bb3c9 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:54.196363 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:54.222483 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed31bf-81a7-498d-b313-2b82bd7bb3c9 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:54.261875 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c77f64-0136-41b2-a1c3-70a7cbb690fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:54.262508 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c77f64-0136-41b2-a1c3-70a7cbb690fd None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:54.262794 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c77f64-0136-41b2-a1c3-70a7cbb690fd None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:54.263074 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c77f64-0136-41b2-a1c3-70a7cbb690fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:54.286182 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebcaa8d-68d5-4f8a-8fd9-1ef2c8121dac None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:54.286801 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebcaa8d-68d5-4f8a-8fd9-1ef2c8121dac None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:54.287272 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebcaa8d-68d5-4f8a-8fd9-1ef2c8121dac None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:54.289249 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 211/428] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:54.294724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795ff17-ae58-4660-9773-01af446a7f02 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:54.295000 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795ff17-ae58-4660-9773-01af446a7f02 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:54.295282 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795ff17-ae58-4660-9773-01af446a7f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:54.319979 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e795ff17-ae58-4660-9773-01af446a7f02 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:54.321802 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e795ff17-ae58-4660-9773-01af446a7f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:54.341440 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c77f64-0136-41b2-a1c3-70a7cbb690fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:54.343002 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c77f64-0136-41b2-a1c3-70a7cbb690fd None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:54.405739 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 212/429] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:55:54.427969 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:54.429018 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e795ff17-ae58-4660-9773-01af446a7f02 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:54.449781 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 218/430] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:55:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:55:54.456283 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c77f64-0136-41b2-a1c3-70a7cbb690fd None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:54.543099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e795ff17-ae58-4660-9773-01af446a7f02 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:54.545040 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:54.565032 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c77f64-0136-41b2-a1c3-70a7cbb690fd None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:54.583699 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400b328-20ca-4044-9091-e3f956903973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:54.584642 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400b328-20ca-4044-9091-e3f956903973 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:54.585102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400b328-20ca-4044-9091-e3f956903973 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:54.585564 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400b328-20ca-4044-9091-e3f956903973 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:54.597356 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f400b328-20ca-4044-9091-e3f956903973 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:54.600643 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f5334-de47-4bb7-be28-5ba1ab987930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:54.601296 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f5334-de47-4bb7-be28-5ba1ab987930 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:54.601582 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f5334-de47-4bb7-be28-5ba1ab987930 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:54.601862 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f5334-de47-4bb7-be28-5ba1ab987930 None admin] PATH_INFO: `/v3/projects/516ee04b71684e5885fa9a08b821fb87` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:54.603034 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743f5334-de47-4bb7-be28-5ba1ab987930 None admin] RBAC: Authorizing `identity:get_project(project_id=516ee04b71684e5885fa9a08b821fb87)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:54.604632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-f400b328-20ca-4044-9091-e3f956903973 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:55:54.604632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:54.604632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:54.604632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:54.604632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:54.604632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:54.604632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:54.604632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:54.604632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:54.604632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:54.604632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:54.604632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:54.604632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:54.604632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:54.604632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:54.604632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:54.605452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:54.605452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:54.605452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:54.605452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:54.605452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:54.605452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:54.605452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:54.605452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:54.605452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:54.605452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:54.605452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:54.605452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:54.605452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:54.605452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:54.605452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:54.605452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:54.606215 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:54.606215 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:54.606215 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:54.606215 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:54.606215 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:54.606215 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:54.606215 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:54.606215 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:54.606215 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:54.606215 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:54.606215 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 13:55:54.606215 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:55:54.606788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743f5334-de47-4bb7-be28-5ba1ab987930 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:54.606788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 219/431] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:54 2022] GET /identity/v3/projects/516ee04b71684e5885fa9a08b821fb87 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:54.611578 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 213/432] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:54.612776 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f5334-de47-4bb7-be28-5ba1ab987930 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:54.614867 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400b328-20ca-4044-9091-e3f956903973 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:54.669578 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:54.726216 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f5334-de47-4bb7-be28-5ba1ab987930 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:54.730216 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400b328-20ca-4044-9091-e3f956903973 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:54.764085 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef7088-6503-40eb-b049-79f87ae8fae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:54.764819 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef7088-6503-40eb-b049-79f87ae8fae7 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:54.765105 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef7088-6503-40eb-b049-79f87ae8fae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:54.765376 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef7088-6503-40eb-b049-79f87ae8fae7 None admin] PATH_INFO: `/v3/groups/158f3c1b59904f1f93ecd8987d832191` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:54.766577 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef7088-6503-40eb-b049-79f87ae8fae7 None admin] RBAC: Authorizing `identity:get_group(group_id=158f3c1b59904f1f93ecd8987d832191)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:54.768028 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef7088-6503-40eb-b049-79f87ae8fae7 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:54.770013 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 220/433] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 13:55:54 2022] GET /identity/v3/groups/158f3c1b59904f1f93ecd8987d832191 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:54.771689 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dedf190-123b-4983-ae9c-1be4d2017678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:54.772308 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dedf190-123b-4983-ae9c-1be4d2017678 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:54.773562 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dedf190-123b-4983-ae9c-1be4d2017678 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:54.773562 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dedf190-123b-4983-ae9c-1be4d2017678 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:54.773767 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dedf190-123b-4983-ae9c-1be4d2017678 None admin] RBAC: Adding query filter params (name=member) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:54.773999 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dedf190-123b-4983-ae9c-1be4d2017678 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:54.775368 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef7088-6503-40eb-b049-79f87ae8fae7 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:54.775479 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dedf190-123b-4983-ae9c-1be4d2017678 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:54.788603 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 214/434] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 13:55:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:54.791217 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dedf190-123b-4983-ae9c-1be4d2017678 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:54.831130 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:54.898321 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef7088-6503-40eb-b049-79f87ae8fae7 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:54.918192 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dedf190-123b-4983-ae9c-1be4d2017678 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:54.918663 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:54.942473 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d356dfba-8809-4f62-9537-a67ae69a625c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:54.943353 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d356dfba-8809-4f62-9537-a67ae69a625c None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:54.943766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d356dfba-8809-4f62-9537-a67ae69a625c None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:54.944275 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d356dfba-8809-4f62-9537-a67ae69a625c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:54.945159 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d356dfba-8809-4f62-9537-a67ae69a625c None admin] RBAC: Adding query filter params (group.id=158f3c1b59904f1f93ecd8987d832191, role.id=47e170213d8049a28f8b8a0d736fdf6b, scope.project.id=516ee04b71684e5885fa9a08b821fb87) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:54.945642 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d356dfba-8809-4f62-9537-a67ae69a625c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:54.947359 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d356dfba-8809-4f62-9537-a67ae69a625c None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:54.955125 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 221/435] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 1 13:55:54 2022] GET /identity/v3/role_assignments?group.id=158f3c1b59904f1f93ecd8987d832191&scope.project.id=516ee04b71684e5885fa9a08b821fb87&role.id=47e170213d8049a28f8b8a0d736fdf6b => generated 611 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:54.955561 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034f7c2-1a82-4298-a2fd-38abd4fb2b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:54.956457 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034f7c2-1a82-4298-a2fd-38abd4fb2b4f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:54.956771 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034f7c2-1a82-4298-a2fd-38abd4fb2b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:54.957049 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034f7c2-1a82-4298-a2fd-38abd4fb2b4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:55.034396 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c034f7c2-1a82-4298-a2fd-38abd4fb2b4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:55.035877 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c034f7c2-1a82-4298-a2fd-38abd4fb2b4f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:55.063247 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:55.120137 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b988621f-4959-47d1-8ce4-05b4bed068c8 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:55.120643 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b988621f-4959-47d1-8ce4-05b4bed068c8 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:55.121034 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b988621f-4959-47d1-8ce4-05b4bed068c8 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:55.122461 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 222/436] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:55:55.129209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f0bc95-a3c2-45bb-8d11-c5f020b068d9 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:55.129844 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f0bc95-a3c2-45bb-8d11-c5f020b068d9 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:55.130198 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f0bc95-a3c2-45bb-8d11-c5f020b068d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:55.150633 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 215/437] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:55:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 13:55:55.151896 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-24f0bc95-a3c2-45bb-8d11-c5f020b068d9 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:55.153135 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-24f0bc95-a3c2-45bb-8d11-c5f020b068d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:55.157403 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034f7c2-1a82-4298-a2fd-38abd4fb2b4f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:55.238991 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 223/438] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:55:55.265182 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f0bc95-a3c2-45bb-8d11-c5f020b068d9 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:55.293744 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034f7c2-1a82-4298-a2fd-38abd4fb2b4f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:55.323392 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:55.337950 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8f2754-3fd1-4af5-a124-a26bc3d6b473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:55.338639 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f2754-3fd1-4af5-a124-a26bc3d6b473 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:55.338930 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f2754-3fd1-4af5-a124-a26bc3d6b473 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:55.339309 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f2754-3fd1-4af5-a124-a26bc3d6b473 None admin] PATH_INFO: `/v3/projects/05e772d8318b47469008ecb063004d71` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:55.340584 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8f2754-3fd1-4af5-a124-a26bc3d6b473 None admin] RBAC: Authorizing `identity:get_project(project_id=05e772d8318b47469008ecb063004d71)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:55.342160 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8f2754-3fd1-4af5-a124-a26bc3d6b473 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:55.344064 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 216/439] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:55 2022] GET /identity/v3/projects/05e772d8318b47469008ecb063004d71 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:55.345236 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3bc2bd-2ce4-4a3d-b54c-f92b0acde740 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:55.345523 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3bc2bd-2ce4-4a3d-b54c-f92b0acde740 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:55.345798 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3bc2bd-2ce4-4a3d-b54c-f92b0acde740 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:55.347005 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 217/440] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:55.350468 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3bc2bd-2ce4-4a3d-b54c-f92b0acde740 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:55.384926 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f0bc95-a3c2-45bb-8d11-c5f020b068d9 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:55.398626 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:55.419724 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad493e20-6486-4076-9e08-b10e2d96d4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:55.420293 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad493e20-6486-4076-9e08-b10e2d96d4be None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:55.420604 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad493e20-6486-4076-9e08-b10e2d96d4be None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:55.420880 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad493e20-6486-4076-9e08-b10e2d96d4be None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:55.430816 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad493e20-6486-4076-9e08-b10e2d96d4be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:55.436445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-ad493e20-6486-4076-9e08-b10e2d96d4be None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:55:55.436445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:55.436445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:55.436445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:55.436445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:55.436445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:55.436445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:55.436445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:55.436445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:55.436445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:55.436445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:55.436445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:55.436445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:55.436445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:55.436445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:55.436445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:55.437327 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:55.437327 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:55.437327 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:55.437327 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:55.437327 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:55.437327 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:55.437327 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:55.437327 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:55.437327 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:55.437327 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:55.437327 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:55.437327 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:55.437327 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:55.437327 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:55.437327 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:55.437327 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:55.438092 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:55.438092 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:55.438092 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:55.438092 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:55.438092 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:55.438092 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:55.438092 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:55.438092 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:55.438092 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:55.438092 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:55.438092 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:55:55.438092 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:55:55.438092 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 224/441] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 13:55:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:55.439118 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07c827-289d-4121-9e1c-381387c04886 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:55.439316 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07c827-289d-4121-9e1c-381387c04886 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:55.439574 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07c827-289d-4121-9e1c-381387c04886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:55.456528 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-4a07c827-289d-4121-9e1c-381387c04886 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:55.457697 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-4a07c827-289d-4121-9e1c-381387c04886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:55.462948 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3bc2bd-2ce4-4a3d-b54c-f92b0acde740 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:55.498287 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ffedea-f8eb-44a7-8875-8810a769efc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:55.498925 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffedea-f8eb-44a7-8875-8810a769efc1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:55.499209 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffedea-f8eb-44a7-8875-8810a769efc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:55.499494 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffedea-f8eb-44a7-8875-8810a769efc1 None admin] PATH_INFO: `/v3/groups/158f3c1b59904f1f93ecd8987d832191` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:55.500704 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ffedea-f8eb-44a7-8875-8810a769efc1 None admin] RBAC: Authorizing `identity:get_group(group_id=158f3c1b59904f1f93ecd8987d832191)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:55.502132 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ffedea-f8eb-44a7-8875-8810a769efc1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:55.503720 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 218/442] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 13:55:55 2022] GET /identity/v3/groups/158f3c1b59904f1f93ecd8987d832191 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:55.504550 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ffedea-f8eb-44a7-8875-8810a769efc1 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:55.533921 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 225/443] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:55.533921 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a07c827-289d-4121-9e1c-381387c04886 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:55.570103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:55.612225 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ffedea-f8eb-44a7-8875-8810a769efc1 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:55.645511 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a07c827-289d-4121-9e1c-381387c04886 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:55.645691 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c300a-b887-401d-b335-b53a1e96d8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:55.646310 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c300a-b887-401d-b335-b53a1e96d8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:55.646593 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c300a-b887-401d-b335-b53a1e96d8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:55.646875 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c300a-b887-401d-b335-b53a1e96d8b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:55.647197 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:55.647587 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c300a-b887-401d-b335-b53a1e96d8b4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:55.647898 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c300a-b887-401d-b335-b53a1e96d8b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:55.649550 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c300a-b887-401d-b335-b53a1e96d8b4 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:55.657649 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 219/444] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:55:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:55.658433 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c300a-b887-401d-b335-b53a1e96d8b4 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:55.682229 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18660c6-fb05-47a7-8c62-b9e569fe9179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:55.682789 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18660c6-fb05-47a7-8c62-b9e569fe9179 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:55.683076 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18660c6-fb05-47a7-8c62-b9e569fe9179 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:55.683357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18660c6-fb05-47a7-8c62-b9e569fe9179 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:55.684153 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18660c6-fb05-47a7-8c62-b9e569fe9179 None admin] RBAC: Adding query filter params (group.id=158f3c1b59904f1f93ecd8987d832191, role.id=926b9f36c24847c582c84af904250abc, scope.project.id=05e772d8318b47469008ecb063004d71) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:55.684468 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18660c6-fb05-47a7-8c62-b9e569fe9179 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:55.685920 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18660c6-fb05-47a7-8c62-b9e569fe9179 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:55.691567 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 226/445] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 1 13:55:55 2022] GET /identity/v3/role_assignments?group.id=158f3c1b59904f1f93ecd8987d832191&scope.project.id=05e772d8318b47469008ecb063004d71&role.id=926b9f36c24847c582c84af904250abc => generated 611 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:55.692335 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18660c6-fb05-47a7-8c62-b9e569fe9179 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:55.769910 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c300a-b887-401d-b335-b53a1e96d8b4 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:55.796054 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:55.811797 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18660c6-fb05-47a7-8c62-b9e569fe9179 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:55.815598 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0451bf4a-d382-4ed7-af67-6a573e201f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:55.815598 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451bf4a-d382-4ed7-af67-6a573e201f9f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:55.815598 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451bf4a-d382-4ed7-af67-6a573e201f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:55.815598 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451bf4a-d382-4ed7-af67-6a573e201f9f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:55.828161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0451bf4a-d382-4ed7-af67-6a573e201f9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:55:55.833739 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-0451bf4a-d382-4ed7-af67-6a573e201f9f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:55:55.833739 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:55.833739 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:55.833739 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:55.833739 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:55.833739 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:55.833739 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:55.833739 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:55.833739 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:55.833739 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:55.833739 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:55.833739 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:55.833739 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:55:55.833739 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:55:55.833739 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:55:55.833739 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:55:55.834632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:55:55.834632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:55:55.834632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:55:55.834632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:55:55.834632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:55.834632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:55.834632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:55.834632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:55.834632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:55.834632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:55.834632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:55.834632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:55.834632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:55.834632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:55.834632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:55.834632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:55.840942 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:55.840942 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:55.840942 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:55.840942 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:55.840942 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:55:55.840942 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:55:55.840942 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:55:55.840942 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:55:55.840942 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:55:55.840942 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:55:55.840942 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:55:55.840942 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:55:55.840942 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 220/446] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 13:55:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:55.841635 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0451bf4a-d382-4ed7-af67-6a573e201f9f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:55.865604 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ebb39-24a1-4565-9831-b2f0febdfcca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:55.866303 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ebb39-24a1-4565-9831-b2f0febdfcca None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:55.867294 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ebb39-24a1-4565-9831-b2f0febdfcca None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:55.867294 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ebb39-24a1-4565-9831-b2f0febdfcca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:55.905609 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:55.951683 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0ebb39-24a1-4565-9831-b2f0febdfcca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:55.953117 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0ebb39-24a1-4565-9831-b2f0febdfcca None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:55.965714 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0451bf4a-d382-4ed7-af67-6a573e201f9f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:56.007545 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1adf302-5de8-496d-8104-4c66c737cbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:56.008176 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1adf302-5de8-496d-8104-4c66c737cbaf None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:56.008466 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1adf302-5de8-496d-8104-4c66c737cbaf None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:56.008775 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1adf302-5de8-496d-8104-4c66c737cbaf None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:56.009540 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1adf302-5de8-496d-8104-4c66c737cbaf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:56.009859 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1adf302-5de8-496d-8104-4c66c737cbaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:56.011303 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1adf302-5de8-496d-8104-4c66c737cbaf None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:56.019733 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 221/447] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:55:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:56.026302 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1adf302-5de8-496d-8104-4c66c737cbaf None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:56.073087 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 227/448] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:55:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:55:56.076438 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:56.078640 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ebb39-24a1-4565-9831-b2f0febdfcca None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:56.130249 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1adf302-5de8-496d-8104-4c66c737cbaf None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:56.141700 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:56.164255 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f931f2d-f25c-4455-a2d9-585c2954c049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:56.164901 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f931f2d-f25c-4455-a2d9-585c2954c049 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:56.165183 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f931f2d-f25c-4455-a2d9-585c2954c049 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:56.165457 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f931f2d-f25c-4455-a2d9-585c2954c049 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:56.177834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ebb39-24a1-4565-9831-b2f0febdfcca None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:56.210727 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6438f228-16a0-4d6f-9234-09e74872fb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:56.211296 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6438f228-16a0-4d6f-9234-09e74872fb42 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:56.211577 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6438f228-16a0-4d6f-9234-09e74872fb42 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:56.211853 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6438f228-16a0-4d6f-9234-09e74872fb42 None admin] PATH_INFO: `/v3/projects/05e772d8318b47469008ecb063004d71` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:56.213715 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6438f228-16a0-4d6f-9234-09e74872fb42 None admin] RBAC: Authorizing `identity:get_project(project_id=05e772d8318b47469008ecb063004d71)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:56.215138 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6438f228-16a0-4d6f-9234-09e74872fb42 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:56.216821 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 228/449] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:56 2022] GET /identity/v3/projects/05e772d8318b47469008ecb063004d71 => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:56.222202 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6438f228-16a0-4d6f-9234-09e74872fb42 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:56.233417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f931f2d-f25c-4455-a2d9-585c2954c049 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:56.234875 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f931f2d-f25c-4455-a2d9-585c2954c049 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:56.280904 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:56.324471 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6438f228-16a0-4d6f-9234-09e74872fb42 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:56.334630 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 222/450] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:55:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 13:55:56.340736 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f931f2d-f25c-4455-a2d9-585c2954c049 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:56.358363 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ccafa-2242-4068-a5dc-7eaa2385913f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:56.359008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ccafa-2242-4068-a5dc-7eaa2385913f None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:56.359296 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ccafa-2242-4068-a5dc-7eaa2385913f None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:56.359579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ccafa-2242-4068-a5dc-7eaa2385913f None admin] PATH_INFO: `/v3/groups/158f3c1b59904f1f93ecd8987d832191` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:56.360788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3ccafa-2242-4068-a5dc-7eaa2385913f None admin] RBAC: Authorizing `identity:get_group(group_id=158f3c1b59904f1f93ecd8987d832191)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:56.362229 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3ccafa-2242-4068-a5dc-7eaa2385913f None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:56.364110 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 229/451] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 13:55:56 2022] GET /identity/v3/groups/158f3c1b59904f1f93ecd8987d832191 => generated 226 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:56.369600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ccafa-2242-4068-a5dc-7eaa2385913f None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:56.372238 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:56.440151 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f931f2d-f25c-4455-a2d9-585c2954c049 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:56.473789 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7e0e1-53a3-498f-bfd6-58ebf5de600e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:56.474324 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ccafa-2242-4068-a5dc-7eaa2385913f None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:56.474428 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7e0e1-53a3-498f-bfd6-58ebf5de600e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:56.474621 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7e0e1-53a3-498f-bfd6-58ebf5de600e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:56.474895 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7e0e1-53a3-498f-bfd6-58ebf5de600e None admin] PATH_INFO: `/v3/projects/b30b7ebe5b2d4e7c8f161e5b78a7a0df` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:56.476047 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a7e0e1-53a3-498f-bfd6-58ebf5de600e None admin] RBAC: Authorizing `identity:get_project(project_id=b30b7ebe5b2d4e7c8f161e5b78a7a0df)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:56.477499 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a7e0e1-53a3-498f-bfd6-58ebf5de600e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:56.479157 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 223/452] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:55:56 2022] GET /identity/v3/projects/b30b7ebe5b2d4e7c8f161e5b78a7a0df => generated 344 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:56.484444 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7e0e1-53a3-498f-bfd6-58ebf5de600e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:56.498594 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:56.511915 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f30189-e335-4df6-b005-8a20939ba73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:56.512574 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f30189-e335-4df6-b005-8a20939ba73c None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:56.512855 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f30189-e335-4df6-b005-8a20939ba73c None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:56.513127 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f30189-e335-4df6-b005-8a20939ba73c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:56.513911 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f30189-e335-4df6-b005-8a20939ba73c None admin] RBAC: Adding query filter params (group.id=158f3c1b59904f1f93ecd8987d832191, role.id=47e170213d8049a28f8b8a0d736fdf6b, scope.project.id=05e772d8318b47469008ecb063004d71) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:56.514222 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f30189-e335-4df6-b005-8a20939ba73c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:56.515653 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f30189-e335-4df6-b005-8a20939ba73c None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:56.521233 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 230/453] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 1 13:55:56 2022] GET /identity/v3/role_assignments?group.id=158f3c1b59904f1f93ecd8987d832191&scope.project.id=05e772d8318b47469008ecb063004d71&role.id=47e170213d8049a28f8b8a0d736fdf6b => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:56.588228 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7e0e1-53a3-498f-bfd6-58ebf5de600e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:56.624081 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2430d5-3529-481c-a8db-45f9b27c97be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:56.624771 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2430d5-3529-481c-a8db-45f9b27c97be None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:56.625052 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2430d5-3529-481c-a8db-45f9b27c97be None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:56.625328 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2430d5-3529-481c-a8db-45f9b27c97be None admin] PATH_INFO: `/v3/groups/f73bbef49520470cb75b98a774a1bcfe` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:56.626632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2430d5-3529-481c-a8db-45f9b27c97be None admin] RBAC: Authorizing `identity:get_group(group_id=f73bbef49520470cb75b98a774a1bcfe)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:56.628115 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2430d5-3529-481c-a8db-45f9b27c97be None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:56.629827 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 224/454] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 13:55:56 2022] GET /identity/v3/groups/f73bbef49520470cb75b98a774a1bcfe => generated 229 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:56.636777 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f30189-e335-4df6-b005-8a20939ba73c None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:56.703855 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:56.738161 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f30189-e335-4df6-b005-8a20939ba73c None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:56.773566 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1fe897-4e2e-431e-b12d-652c00b1c6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:56.774182 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1fe897-4e2e-431e-b12d-652c00b1c6ef None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:56.774465 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1fe897-4e2e-431e-b12d-652c00b1c6ef None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:56.774742 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1fe897-4e2e-431e-b12d-652c00b1c6ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:56.775565 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1fe897-4e2e-431e-b12d-652c00b1c6ef None admin] RBAC: Adding query filter params (group.id=f73bbef49520470cb75b98a774a1bcfe, role.id=58f91e3a13664dabb18c6227a1658fa6, scope.project.id=b30b7ebe5b2d4e7c8f161e5b78a7a0df) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:56.775883 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1fe897-4e2e-431e-b12d-652c00b1c6ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:56.777351 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1fe897-4e2e-431e-b12d-652c00b1c6ef None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:56.783279 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 231/455] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 1 13:55:56 2022] GET /identity/v3/role_assignments?group.id=f73bbef49520470cb75b98a774a1bcfe&scope.project.id=b30b7ebe5b2d4e7c8f161e5b78a7a0df&role.id=58f91e3a13664dabb18c6227a1658fa6 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:55:59.220524 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36058c-c0ac-4c8d-9df2-4a547afb536e None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:59.221070 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36058c-c0ac-4c8d-9df2-4a547afb536e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:59.221148 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36058c-c0ac-4c8d-9df2-4a547afb536e None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:59.222805 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 225/456] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:59.228192 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ad220-6702-4f3c-aba4-132ad947dab5 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:59.228662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ad220-6702-4f3c-aba4-132ad947dab5 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:59.228761 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ad220-6702-4f3c-aba4-132ad947dab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:59.252168 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-e64ad220-6702-4f3c-aba4-132ad947dab5 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:59.253396 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-e64ad220-6702-4f3c-aba4-132ad947dab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:59.343152 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 232/457] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:59.379653 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c36058c-c0ac-4c8d-9df2-4a547afb536e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:59.495426 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:59.526735 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c36058c-c0ac-4c8d-9df2-4a547afb536e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:59.574119 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b204c31-fefd-4085-8413-4a809fa38912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:59.575121 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b204c31-fefd-4085-8413-4a809fa38912 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:59.575631 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b204c31-fefd-4085-8413-4a809fa38912 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:59.576150 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b204c31-fefd-4085-8413-4a809fa38912 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:59.579308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6254707-4fc6-4a7e-9d58-13a3e8f5d08e None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:59.579591 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6254707-4fc6-4a7e-9d58-13a3e8f5d08e None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:59.579876 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6254707-4fc6-4a7e-9d58-13a3e8f5d08e None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:59.582208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 233/458] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:59.588513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17daf376-84ea-4b13-8d2c-5e134df88d10 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:59.588513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17daf376-84ea-4b13-8d2c-5e134df88d10 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:59.588513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17daf376-84ea-4b13-8d2c-5e134df88d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:59.588996 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b204c31-fefd-4085-8413-4a809fa38912 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:59.590474 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b204c31-fefd-4085-8413-4a809fa38912 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:59.607228 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-5b204c31-fefd-4085-8413-4a809fa38912 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:55:59.607228 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:55:59.607228 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:55:59.607228 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:55:59.607228 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:55:59.607228 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:55:59.607228 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:55:59.607228 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:55:59.607228 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:55:59.607228 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:55:59.607228 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:55:59.607228 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:55:59.607228 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:55:59.607228 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:55:59.607228 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:55:59.608156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:55:59.608156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:55:59.608156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:55:59.608156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:55:59.608156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:55:59.608156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:55:59.608156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:55:59.608156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:55:59.608156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:55:59.608156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:55:59.608156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:55:59.608156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:55:59.608156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:55:59.608156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:55:59.608156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:55:59.608156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:55:59.608156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:55:59.609110 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:55:59.609110 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:55:59.609110 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:55:59.609110 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:55:59.609110 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 226/459] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:55:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:55:59.614398 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b204c31-fefd-4085-8413-4a809fa38912 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:59.625964 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-17daf376-84ea-4b13-8d2c-5e134df88d10 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:59.627154 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-17daf376-84ea-4b13-8d2c-5e134df88d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:59.739693 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:59.756832 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 234/460] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:55:59.758401 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d0a10-0400-4def-83db-3cd745a4af6e None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:59.758690 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d0a10-0400-4def-83db-3cd745a4af6e None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:59.758969 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d0a10-0400-4def-83db-3cd745a4af6e None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:59.760891 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 235/461] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:59.770662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1157074-acb4-41b9-90a6-b861dfd70b1b None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:59.770949 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1157074-acb4-41b9-90a6-b861dfd70b1b None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:59.771226 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1157074-acb4-41b9-90a6-b861dfd70b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:59.788642 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b204c31-fefd-4085-8413-4a809fa38912 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:59.805306 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-c1157074-acb4-41b9-90a6-b861dfd70b1b None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:55:59.807029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-c1157074-acb4-41b9-90a6-b861dfd70b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:55:59.824793 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:55:59.830335 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d04e86-82a7-4fce-af88-6e8d1048bb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:55:59.830963 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d04e86-82a7-4fce-af88-6e8d1048bb37 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:59.831315 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d04e86-82a7-4fce-af88-6e8d1048bb37 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:59.831657 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d04e86-82a7-4fce-af88-6e8d1048bb37 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:59.832533 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d04e86-82a7-4fce-af88-6e8d1048bb37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:55:59.833473 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d04e86-82a7-4fce-af88-6e8d1048bb37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:55:59.835624 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d04e86-82a7-4fce-af88-6e8d1048bb37 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:55:59.856572 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 227/462] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:55:59.857666 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d04e86-82a7-4fce-af88-6e8d1048bb37 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:59.885611 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 236/463] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:55:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:55:59.886861 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0106e6-cc64-4022-8aa1-e094310f5648 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:55:59.887147 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0106e6-cc64-4022-8aa1-e094310f5648 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:55:59.887422 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0106e6-cc64-4022-8aa1-e094310f5648 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:55:59.892643 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 237/464] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:55:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:55:59.892643 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0106e6-cc64-4022-8aa1-e094310f5648 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:55:59.969662 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d04e86-82a7-4fce-af88-6e8d1048bb37 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:55:59.996934 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0106e6-cc64-4022-8aa1-e094310f5648 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:00.006573 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc496102-23b1-4726-ab0f-5adb72917e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:00.007881 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc496102-23b1-4726-ab0f-5adb72917e46 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:00.008370 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc496102-23b1-4726-ab0f-5adb72917e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:00.008829 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc496102-23b1-4726-ab0f-5adb72917e46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:00.017449 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:00.021835 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc496102-23b1-4726-ab0f-5adb72917e46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:00.023466 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc496102-23b1-4726-ab0f-5adb72917e46 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:00.033026 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8236409f-23e7-49df-9ea1-2ae11a2ef150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:00.033657 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8236409f-23e7-49df-9ea1-2ae11a2ef150 None admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:00.033924 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8236409f-23e7-49df-9ea1-2ae11a2ef150 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:00.034202 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8236409f-23e7-49df-9ea1-2ae11a2ef150 None admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:00.034881 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8236409f-23e7-49df-9ea1-2ae11a2ef150 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:00.036300 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8236409f-23e7-49df-9ea1-2ae11a2ef150 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:00.041289 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-bc496102-23b1-4726-ab0f-5adb72917e46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:00.041289 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:00.041289 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:00.041289 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:00.041289 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:00.041289 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:00.041289 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:00.041289 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:00.041289 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:00.041289 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:00.041289 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:00.041289 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:00.041289 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:00.041289 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:00.041289 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:00.042085 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:00.042085 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:00.042085 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:00.042085 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:00.042085 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:00.042085 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:00.042085 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:00.042085 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:00.042085 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:00.042085 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:00.042085 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:00.042085 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:00.042085 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:00.042085 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:00.042085 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:00.042085 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:00.042085 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:00.042968 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:00.042968 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:00.042968 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:00.042968 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:00.044037 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 228/465] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:55:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:00.045283 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80591e67-f9bd-4614-b392-1731c1397cfe None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:00.045574 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80591e67-f9bd-4614-b392-1731c1397cfe None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:00.045854 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80591e67-f9bd-4614-b392-1731c1397cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:00.072542 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 238/466] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 13:55:59 2022] POST /identity/v3/users => generated 246 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:56:00.072750 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-80591e67-f9bd-4614-b392-1731c1397cfe None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:00.072841 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8236409f-23e7-49df-9ea1-2ae11a2ef150 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:00.072947 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-80591e67-f9bd-4614-b392-1731c1397cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:00.121902 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:00.156522 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 229/467] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:56:00.157583 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80591e67-f9bd-4614-b392-1731c1397cfe None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:00.221436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8236409f-23e7-49df-9ea1-2ae11a2ef150 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:00.269034 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c479b-553c-4428-8ce1-358dd3df3e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:00.269630 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c479b-553c-4428-8ce1-358dd3df3e5d None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:00.269630 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c479b-553c-4428-8ce1-358dd3df3e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:00.269901 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c479b-553c-4428-8ce1-358dd3df3e5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:00.275248 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80591e67-f9bd-4614-b392-1731c1397cfe None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:00.284573 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c479b-553c-4428-8ce1-358dd3df3e5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:00.286230 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c479b-553c-4428-8ce1-358dd3df3e5d None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:00.303022 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-b61c479b-553c-4428-8ce1-358dd3df3e5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:00.303022 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:00.303022 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:00.303022 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:00.303022 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:00.303022 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:00.303022 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:00.303022 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:00.303022 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:00.303022 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:00.303022 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:00.303022 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:00.303022 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:00.303022 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:00.303022 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:00.303933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:00.303933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:00.303933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:00.303933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:00.303933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:00.303933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:00.303933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:00.303933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:00.303933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:00.303933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:00.303933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:00.303933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:00.303933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:00.303933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:00.303933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:00.303933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:00.303933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:00.304809 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:00.304809 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:00.304809 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:00.304809 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:00.304809 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 239/468] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:00.305078 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c479b-553c-4428-8ce1-358dd3df3e5d None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:00.328042 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2b239f-1011-419b-bf73-36ee1809eeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:00.328705 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b239f-1011-419b-bf73-36ee1809eeea None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:00.328980 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b239f-1011-419b-bf73-36ee1809eeea None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:00.329274 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b239f-1011-419b-bf73-36ee1809eeea None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:00.329980 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2b239f-1011-419b-bf73-36ee1809eeea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:00.330284 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2b239f-1011-419b-bf73-36ee1809eeea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:00.331657 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2b239f-1011-419b-bf73-36ee1809eeea None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:00.344369 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 230/469] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:00.345221 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2b239f-1011-419b-bf73-36ee1809eeea None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:00.352034 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:00.385517 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:00.436691 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c479b-553c-4428-8ce1-358dd3df3e5d None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:00.472378 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2b239f-1011-419b-bf73-36ee1809eeea None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:00.475231 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115b4d0-6e04-4657-b56c-85070a381ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:00.476077 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115b4d0-6e04-4657-b56c-85070a381ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:00.476529 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115b4d0-6e04-4657-b56c-85070a381ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:00.476949 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115b4d0-6e04-4657-b56c-85070a381ff2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:00.488696 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7115b4d0-6e04-4657-b56c-85070a381ff2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:00.490419 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7115b4d0-6e04-4657-b56c-85070a381ff2 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:00.510858 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-7115b4d0-6e04-4657-b56c-85070a381ff2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:00.510858 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:00.510858 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:00.510858 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:00.510858 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:00.510858 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:00.510858 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:00.510858 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:00.510858 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:00.510858 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:00.510858 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:00.510858 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:00.510858 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:00.510858 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:00.510858 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:00.512007 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:00.512007 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:00.512007 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:00.512007 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:00.512007 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:00.512007 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:00.512007 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:00.512007 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:00.512007 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:00.512007 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:00.512007 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:00.512007 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:00.512007 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:00.512007 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:00.512007 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:00.512007 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:00.512007 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:00.512993 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:00.512993 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:00.512993 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:00.512993 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:00.513201 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1bf46-2247-444d-9ee0-58a3b3230148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:00.513201 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1bf46-2247-444d-9ee0-58a3b3230148 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:00.513201 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1bf46-2247-444d-9ee0-58a3b3230148 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:00.513201 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1bf46-2247-444d-9ee0-58a3b3230148 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:00.513461 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be1bf46-2247-444d-9ee0-58a3b3230148 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:00.513461 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be1bf46-2247-444d-9ee0-58a3b3230148 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:00.514475 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be1bf46-2247-444d-9ee0-58a3b3230148 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:00.517937 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 240/470] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:00.518652 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115b4d0-6e04-4657-b56c-85070a381ff2 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:00.528386 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 231/471] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:00.533944 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1bf46-2247-444d-9ee0-58a3b3230148 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:00.622642 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:00.643804 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115b4d0-6e04-4657-b56c-85070a381ff2 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:00.645559 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:00.655216 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1bf46-2247-444d-9ee0-58a3b3230148 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:00.679918 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff7216-3772-4568-9845-365f85f85adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:00.680466 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff7216-3772-4568-9845-365f85f85adb None admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:00.680777 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff7216-3772-4568-9845-365f85f85adb None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:00.681120 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff7216-3772-4568-9845-365f85f85adb None admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:00.681802 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beff7216-3772-4568-9845-365f85f85adb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:00.683250 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beff7216-3772-4568-9845-365f85f85adb None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:00.693614 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cfd88f-d5ac-467d-9b24-4e6691d6f5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:00.695946 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfd88f-d5ac-467d-9b24-4e6691d6f5e3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:00.695946 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfd88f-d5ac-467d-9b24-4e6691d6f5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:00.695946 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfd88f-d5ac-467d-9b24-4e6691d6f5e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:00.695946 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cfd88f-d5ac-467d-9b24-4e6691d6f5e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:00.695946 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cfd88f-d5ac-467d-9b24-4e6691d6f5e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:00.697368 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cfd88f-d5ac-467d-9b24-4e6691d6f5e3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:00.712614 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 241/472] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 13:56:00 2022] POST /identity/v3/users => generated 248 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:56:00.712614 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff7216-3772-4568-9845-365f85f85adb None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:00.717745 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 232/473] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:00.719167 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07bac84-ece8-4fc3-a194-350d53af9fc0 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:00.719636 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07bac84-ece8-4fc3-a194-350d53af9fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:00.719990 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07bac84-ece8-4fc3-a194-350d53af9fc0 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:00.721696 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 233/474] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:00.726738 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07bac84-ece8-4fc3-a194-350d53af9fc0 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:00.849491 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff7216-3772-4568-9845-365f85f85adb None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:00.853598 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07bac84-ece8-4fc3-a194-350d53af9fc0 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:00.895330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67977656-1a05-4f0f-af1d-c007bef00539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:00.896131 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67977656-1a05-4f0f-af1d-c007bef00539 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:00.896510 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67977656-1a05-4f0f-af1d-c007bef00539 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:00.896881 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67977656-1a05-4f0f-af1d-c007bef00539 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:00.898410 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67977656-1a05-4f0f-af1d-c007bef00539 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:00.900466 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67977656-1a05-4f0f-af1d-c007bef00539 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:00.900857 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad81214-3238-44bb-b4f7-f5cb0d668aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:00.901643 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad81214-3238-44bb-b4f7-f5cb0d668aaf None admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:00.902088 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad81214-3238-44bb-b4f7-f5cb0d668aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:00.902508 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad81214-3238-44bb-b4f7-f5cb0d668aaf None admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:00.903328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad81214-3238-44bb-b4f7-f5cb0d668aaf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:00.904998 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad81214-3238-44bb-b4f7-f5cb0d668aaf None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:00.917677 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:00.936509 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 234/475] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 13:56:00 2022] POST /identity/v3/users => generated 249 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:56:00.936509 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f787b6-fd48-4b5a-81fe-6e4c8aef113e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:00.936509 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f787b6-fd48-4b5a-81fe-6e4c8aef113e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:00.936509 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f787b6-fd48-4b5a-81fe-6e4c8aef113e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:00.937404 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 242/476] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 13:56:00 2022] POST /identity/v3/users => generated 248 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:56:00.953724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-73f787b6-fd48-4b5a-81fe-6e4c8aef113e None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:00.955180 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-73f787b6-fd48-4b5a-81fe-6e4c8aef113e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:00.958901 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:01.047730 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 235/477] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:56:01.077599 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad81214-3238-44bb-b4f7-f5cb0d668aaf None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:01.194844 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad81214-3238-44bb-b4f7-f5cb0d668aaf None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:01.232103 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf4ba28-e936-48da-b629-c79701f54094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:01.232947 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4ba28-e936-48da-b629-c79701f54094 None admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:01.233334 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4ba28-e936-48da-b629-c79701f54094 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:01.233713 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4ba28-e936-48da-b629-c79701f54094 None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:01.234546 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf4ba28-e936-48da-b629-c79701f54094 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:01.236014 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf4ba28-e936-48da-b629-c79701f54094 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:01.245000 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.sql.core [None req-7bf4ba28-e936-48da-b629-c79701f54094 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 13:56:01.245000 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 01 13:56:01.245000 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [parameters: {'id': '77d993b122c84215855f1ad6d411dd80', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 01 13:56:01.245000 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80140) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 13:56:01.263709 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-7bf4ba28-e936-48da-b629-c79701f54094 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 01 13:56:01.263709 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:01.263709 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 13:56:01.263709 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 13:56:01.263709 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 13:56:01.263709 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 13:56:01.263709 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 13:56:01.263709 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 13:56:01.263709 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 13:56:01.263709 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application conn.query(q) Aug 01 13:56:01.263709 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 13:56:01.263709 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 13:56:01.263709 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 13:56:01.263709 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application result.read() Aug 01 13:56:01.263709 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 13:56:01.264914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 13:56:01.264914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 13:56:01.264914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 13:56:01.264914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 13:56:01.264914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 13:56:01.264914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 13:56:01.264914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 13:56:01.264914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 13:56:01.264914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:01.264914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 13:56:01.264914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:01.264914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:01.264914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 13:56:01.264914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 13:56:01.264914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 01 13:56:01.264914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return ref.to_dict() Aug 01 13:56:01.264914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 13:56:01.264914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application next(self.gen) Aug 01 13:56:01.266008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 13:56:01.266008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application yield resource Aug 01 13:56:01.266008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 13:56:01.266008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application next(self.gen) Aug 01 13:56:01.266008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 13:56:01.266008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 13:56:01.266008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 13:56:01.266008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 13:56:01.266008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 13:56:01.266008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise self.value Aug 01 13:56:01.266008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 13:56:01.266008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 13:56:01.266008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 13:56:01.266008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application session.commit() Aug 01 13:56:01.266008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 13:56:01.266008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 13:56:01.266008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 13:56:01.266874 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 13:56:01.266874 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 13:56:01.266874 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self.session.flush() Aug 01 13:56:01.266874 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 13:56:01.266874 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 13:56:01.266874 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 13:56:01.266874 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 13:56:01.266874 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 13:56:01.266874 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application compat.raise_( Aug 01 13:56:01.266874 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 13:56:01.266874 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception Aug 01 13:56:01.266874 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 13:56:01.266874 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 13:56:01.266874 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 13:56:01.266874 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 13:56:01.266874 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 13:56:01.266874 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 13:56:01.268292 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 13:56:01.268292 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 13:56:01.268292 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 01 13:56:01.268292 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 01 13:56:01.268292 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 13:56:01.268292 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 13:56:01.268292 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 13:56:01.268292 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 13:56:01.268292 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 13:56:01.268292 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 13:56:01.268292 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 13:56:01.268292 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 13:56:01.268292 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 13:56:01.268292 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 13:56:01.268292 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 13:56:01.268292 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception Aug 01 13:56:01.270308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 13:56:01.270308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 13:56:01.270308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 13:56:01.270308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 13:56:01.270308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 13:56:01.270308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 13:56:01.270308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 13:56:01.270308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application conn.query(q) Aug 01 13:56:01.270308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 13:56:01.270308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 13:56:01.270308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 13:56:01.270308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application result.read() Aug 01 13:56:01.270308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 13:56:01.270308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 13:56:01.270308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 13:56:01.270308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 13:56:01.272071 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 13:56:01.272071 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 13:56:01.272071 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 13:56:01.272071 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 13:56:01.272071 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 13:56:01.272071 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 01 13:56:01.272071 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [parameters: {'id': '77d993b122c84215855f1ad6d411dd80', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 01 13:56:01.272071 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 13:56:01.272071 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:01.272071 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:56:01.272071 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:01.272071 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:01.272071 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:01.272071 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:01.272071 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:01.272071 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:01.276898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:01.276898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:01.276898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:01.276898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:01.276898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:01.276898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:01.276898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 01 13:56:01.276898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 01 13:56:01.276898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:01.276898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:01.276898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 01 13:56:01.276898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 01 13:56:01.276898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 13:56:01.276898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 13:56:01.276898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 01 13:56:01.276898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:01.278860 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 243/478] 10.222.0.48 () {60 vars in 1131 bytes} [Mon Aug 1 13:56:01 2022] POST /identity/v3/roles => generated 144 bytes in 190 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:01.278995 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f787b6-fd48-4b5a-81fe-6e4c8aef113e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:01.394016 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f787b6-fd48-4b5a-81fe-6e4c8aef113e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:01.435331 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4cc0f7-77b2-428d-b4c5-b62fae973893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:01.435988 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4cc0f7-77b2-428d-b4c5-b62fae973893 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:01.436274 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4cc0f7-77b2-428d-b4c5-b62fae973893 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:01.436583 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4cc0f7-77b2-428d-b4c5-b62fae973893 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:01.444987 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:01.450236 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4cc0f7-77b2-428d-b4c5-b62fae973893 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:56:01.455294 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-eb4cc0f7-77b2-428d-b4c5-b62fae973893 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:56:01.455294 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:01.455294 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:01.455294 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:01.455294 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:01.455294 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:01.455294 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:01.455294 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:01.455294 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:01.455294 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:01.455294 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:01.455294 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:01.455294 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:56:01.455294 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:56:01.455294 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:56:01.455294 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:56:01.457449 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:56:01.457449 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:56:01.457449 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:56:01.457449 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:56:01.457449 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:01.457449 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:01.457449 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:01.457449 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:01.457449 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:01.457449 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:01.457449 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:01.457449 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:01.457449 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:01.457449 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:01.457449 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:01.457449 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:01.459464 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:01.459464 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:01.459464 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:01.459464 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:01.459464 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:56:01.459464 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:56:01.459464 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:56:01.459464 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:56:01.459464 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:56:01.459464 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:56:01.459464 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 13:56:01.459464 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:01.459464 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 236/479] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 13:56:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:01.463058 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf4ba28-e936-48da-b629-c79701f54094 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:01.556850 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:01.620369 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf4ba28-e936-48da-b629-c79701f54094 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:01.664154 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201909d6-ff3e-4c92-9df8-21dbfe68d58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:01.664893 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201909d6-ff3e-4c92-9df8-21dbfe68d58f None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:01.665242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201909d6-ff3e-4c92-9df8-21dbfe68d58f None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:01.665526 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201909d6-ff3e-4c92-9df8-21dbfe68d58f None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:01.666296 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201909d6-ff3e-4c92-9df8-21dbfe68d58f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:01.666604 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201909d6-ff3e-4c92-9df8-21dbfe68d58f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:01.668173 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201909d6-ff3e-4c92-9df8-21dbfe68d58f None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:01.680195 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 244/480] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:56:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:02.549899 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa3091-3879-429e-ae40-d73dc06934cf None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:02.550439 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa3091-3879-429e-ae40-d73dc06934cf None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:02.550439 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa3091-3879-429e-ae40-d73dc06934cf None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:02.552056 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 237/481] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:02.575081 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f77794-9bd8-433e-aef0-828ef10bf404 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:02.575594 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f77794-9bd8-433e-aef0-828ef10bf404 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:02.575682 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f77794-9bd8-433e-aef0-828ef10bf404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:02.601145 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-65f77794-9bd8-433e-aef0-828ef10bf404 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:02.602354 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-65f77794-9bd8-433e-aef0-828ef10bf404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:02.695654 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 245/482] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:56:02.729679 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa3091-3879-429e-ae40-d73dc06934cf None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:02.884285 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa3091-3879-429e-ae40-d73dc06934cf None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:02.934074 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc68ece-5f95-411c-976a-bdcdbee8f0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:02.935075 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc68ece-5f95-411c-976a-bdcdbee8f0b0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:02.935581 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc68ece-5f95-411c-976a-bdcdbee8f0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:02.936066 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc68ece-5f95-411c-976a-bdcdbee8f0b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:02.952343 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc68ece-5f95-411c-976a-bdcdbee8f0b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:56:02.958172 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-cdc68ece-5f95-411c-976a-bdcdbee8f0b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:56:02.958172 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:02.958172 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:02.958172 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:02.958172 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:02.958172 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:02.958172 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:02.958172 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:02.958172 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:02.958172 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:02.958172 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:02.958172 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:02.958172 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:56:02.958172 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:56:02.958172 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:56:02.958172 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:56:02.959048 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:56:02.959048 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:56:02.959048 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:56:02.959048 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:56:02.959048 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:02.959048 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:02.959048 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:02.959048 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:02.959048 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:02.959048 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:02.959048 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:02.959048 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:02.959048 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:02.959048 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:02.959048 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:02.959048 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:02.959846 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:02.959846 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:02.959846 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:02.959846 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:02.959846 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:56:02.959846 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:56:02.959846 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:56:02.959846 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:56:02.959846 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:56:02.959846 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:56:02.959846 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:56:02.959846 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:02.967533 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 238/483] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 13:56:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:02.969109 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f77794-9bd8-433e-aef0-828ef10bf404 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:03.048248 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:03.109356 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f77794-9bd8-433e-aef0-828ef10bf404 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:03.162293 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db0e4d-fea0-496f-93d8-e35d5e8448cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:03.162928 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db0e4d-fea0-496f-93d8-e35d5e8448cc None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:03.163215 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db0e4d-fea0-496f-93d8-e35d5e8448cc None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:03.163497 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db0e4d-fea0-496f-93d8-e35d5e8448cc None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:03.164233 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db0e4d-fea0-496f-93d8-e35d5e8448cc None admin] RBAC: Adding query filter params (name=service) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:03.164563 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db0e4d-fea0-496f-93d8-e35d5e8448cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:03.166645 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db0e4d-fea0-496f-93d8-e35d5e8448cc None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:03.179668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 246/484] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 13:56:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:03.185313 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc68ece-5f95-411c-976a-bdcdbee8f0b0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:03.273264 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86feb285-d51a-498d-a5a1-8298b6f426e9 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:03.273619 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86feb285-d51a-498d-a5a1-8298b6f426e9 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:03.273961 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86feb285-d51a-498d-a5a1-8298b6f426e9 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:03.276124 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 247/485] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:03.281091 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5b2f6-942e-43a5-966e-cefd103c465e None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:03.281385 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5b2f6-942e-43a5-966e-cefd103c465e None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:03.281670 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5b2f6-942e-43a5-966e-cefd103c465e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:03.286697 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:03.308902 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-f4b5b2f6-942e-43a5-966e-cefd103c465e None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:03.310237 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-f4b5b2f6-942e-43a5-966e-cefd103c465e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:03.364274 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc68ece-5f95-411c-976a-bdcdbee8f0b0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:03.429242 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e44fad3-e0e7-4f5b-ab42-9735ce618296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:03.429915 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44fad3-e0e7-4f5b-ab42-9735ce618296 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:03.430201 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44fad3-e0e7-4f5b-ab42-9735ce618296 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:03.430476 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44fad3-e0e7-4f5b-ab42-9735ce618296 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:03.445822 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 248/486] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:56:03.450300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e44fad3-e0e7-4f5b-ab42-9735ce618296 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:03.451783 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e44fad3-e0e7-4f5b-ab42-9735ce618296 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:03.467023 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-8e44fad3-e0e7-4f5b-ab42-9735ce618296 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:03.467023 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:03.467023 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:03.467023 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:03.467023 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:03.467023 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:03.467023 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:03.467023 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:03.467023 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:03.467023 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:03.467023 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:03.467023 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:03.467023 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:03.467023 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:03.467023 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:03.467884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:03.467884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:03.467884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:03.467884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:03.467884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:03.467884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:03.467884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:03.467884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:03.467884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:03.467884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:03.467884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:03.467884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:03.467884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:03.467884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:03.467884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:03.467884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:03.467884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:03.469198 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:03.469198 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:03.469198 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:03.469198 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:03.469198 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 239/487] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:03.474041 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5b2f6-942e-43a5-966e-cefd103c465e None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:03.488297 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e44fad3-e0e7-4f5b-ab42-9735ce618296 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:03.548825 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:03.625954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e44fad3-e0e7-4f5b-ab42-9735ce618296 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:03.628425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5b2f6-942e-43a5-966e-cefd103c465e None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:03.641020 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:03.663414 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb14080-3ace-4069-999f-c12c069f9137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:03.664325 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb14080-3ace-4069-999f-c12c069f9137 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:03.664896 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb14080-3ace-4069-999f-c12c069f9137 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:03.665126 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27babaf-4806-48a5-8768-403793e65dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:03.665607 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb14080-3ace-4069-999f-c12c069f9137 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:03.665779 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27babaf-4806-48a5-8768-403793e65dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:03.666022 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27babaf-4806-48a5-8768-403793e65dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:03.666299 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27babaf-4806-48a5-8768-403793e65dd4 None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:03.667054 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27babaf-4806-48a5-8768-403793e65dd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:03.667366 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27babaf-4806-48a5-8768-403793e65dd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:03.668834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27babaf-4806-48a5-8768-403793e65dd4 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:03.677404 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb14080-3ace-4069-999f-c12c069f9137 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:56:03.680266 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 249/488] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:03.683565 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-4bb14080-3ace-4069-999f-c12c069f9137 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:56:03.683565 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:03.683565 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:03.683565 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:03.683565 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:03.683565 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:03.683565 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:03.683565 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:03.683565 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:03.683565 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:03.683565 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:03.683565 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:03.683565 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:56:03.683565 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:56:03.683565 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:56:03.683565 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:56:03.684415 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:56:03.684415 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:56:03.684415 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:56:03.684415 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:56:03.684415 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:03.684415 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:03.684415 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:03.684415 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:03.684415 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:03.684415 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:03.684415 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:03.684415 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:03.684415 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:03.684415 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:03.684415 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:03.684415 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:03.685226 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:03.685226 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:03.685226 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:03.685226 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:03.685226 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:56:03.685226 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:56:03.685226 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:56:03.685226 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:56:03.685226 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:56:03.685226 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:56:03.685226 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:56:03.685226 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:03.686485 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7c950-1468-4b40-af98-1fb959b37d30 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:03.686657 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7c950-1468-4b40-af98-1fb959b37d30 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:03.686916 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7c950-1468-4b40-af98-1fb959b37d30 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:03.692244 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 250/489] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:03.692244 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a17db-8740-4bc2-9813-20d6488032f5 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:03.692438 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 240/490] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 13:56:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:03.692533 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a17db-8740-4bc2-9813-20d6488032f5 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:03.692533 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a17db-8740-4bc2-9813-20d6488032f5 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:03.692655 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb14080-3ace-4069-999f-c12c069f9137 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:03.692720 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 251/491] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:03.692720 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a17db-8740-4bc2-9813-20d6488032f5 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:03.796634 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:03.803226 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a17db-8740-4bc2-9813-20d6488032f5 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:03.815509 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb14080-3ace-4069-999f-c12c069f9137 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:03.840873 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39d792-99e2-4332-b20b-5ff7a063a6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:03.841656 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39d792-99e2-4332-b20b-5ff7a063a6c2 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:03.842859 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39d792-99e2-4332-b20b-5ff7a063a6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:03.843327 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39d792-99e2-4332-b20b-5ff7a063a6c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:03.844143 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a39d792-99e2-4332-b20b-5ff7a063a6c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:03.844458 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a39d792-99e2-4332-b20b-5ff7a063a6c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:03.845870 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a39d792-99e2-4332-b20b-5ff7a063a6c2 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:03.851593 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e52130-ec25-4fdd-84d7-64fa961a48e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:03.852235 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e52130-ec25-4fdd-84d7-64fa961a48e5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:03.853065 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e52130-ec25-4fdd-84d7-64fa961a48e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:03.853350 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e52130-ec25-4fdd-84d7-64fa961a48e5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:03.861841 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 252/492] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 13:56:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:03.862197 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e52130-ec25-4fdd-84d7-64fa961a48e5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:03.862491 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65752f1d-5cf6-4e36-b4a5-68b46dfc544a None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:03.862779 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65752f1d-5cf6-4e36-b4a5-68b46dfc544a None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:03.863058 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65752f1d-5cf6-4e36-b4a5-68b46dfc544a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:03.863744 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e52130-ec25-4fdd-84d7-64fa961a48e5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:03.876532 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-99e52130-ec25-4fdd-84d7-64fa961a48e5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 13:56:03.876532 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:03.876532 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:03.876532 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:03.876532 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:03.876532 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:03.876532 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:03.876532 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:03.876532 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:03.876532 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:03.876532 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:03.876532 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:03.876532 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:56:03.876532 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:56:03.876532 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:56:03.876532 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:56:03.877467 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:03.877467 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:03.877467 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:56:03.877467 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:03.877467 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:56:03.877467 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:03.877467 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:03.877467 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:03.877467 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:03.877467 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:03.877467 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:03.877467 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:03.877467 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:03.877467 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:03.877467 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:03.877467 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:03.877467 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:03.877467 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:03.878348 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:03.878348 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:03.878348 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:56:03.878348 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:56:03.878348 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:56:03.878348 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:56:03.878348 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:56:03.878348 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:56:03.878348 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 13:56:03.878348 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:03.880836 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 241/493] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Aug 1 13:56:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:03.881165 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-65752f1d-5cf6-4e36-b4a5-68b46dfc544a None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:03.882873 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-65752f1d-5cf6-4e36-b4a5-68b46dfc544a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:03.883092 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd01aca8-3bc9-47b3-90c7-c0832e1491d8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:03.883388 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd01aca8-3bc9-47b3-90c7-c0832e1491d8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:03.883740 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd01aca8-3bc9-47b3-90c7-c0832e1491d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:03.902093 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-fd01aca8-3bc9-47b3-90c7-c0832e1491d8 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:03.903330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fd01aca8-3bc9-47b3-90c7-c0832e1491d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:03.931184 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:03.952082 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 253/494] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:56:03.953342 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65752f1d-5cf6-4e36-b4a5-68b46dfc544a None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:03.982903 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 242/495] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:56:03.983748 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd01aca8-3bc9-47b3-90c7-c0832e1491d8 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:04.060387 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:04.069409 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65752f1d-5cf6-4e36-b4a5-68b46dfc544a None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:04.096153 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd01aca8-3bc9-47b3-90c7-c0832e1491d8 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:04.105990 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c427f17-7f33-438a-bd65-922da6a1a824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:04.106644 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c427f17-7f33-438a-bd65-922da6a1a824 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:04.107297 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c427f17-7f33-438a-bd65-922da6a1a824 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:04.107583 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c427f17-7f33-438a-bd65-922da6a1a824 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:04.118820 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c427f17-7f33-438a-bd65-922da6a1a824 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:04.120268 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c427f17-7f33-438a-bd65-922da6a1a824 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:04.132212 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b0df9-2b0b-4258-9ce1-8970c03494cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:04.132794 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b0df9-2b0b-4258-9ce1-8970c03494cb None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:04.133079 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b0df9-2b0b-4258-9ce1-8970c03494cb None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:04.133350 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b0df9-2b0b-4258-9ce1-8970c03494cb None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:04.134672 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b0df9-2b0b-4258-9ce1-8970c03494cb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:04.134978 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b0df9-2b0b-4258-9ce1-8970c03494cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:04.135809 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-6c427f17-7f33-438a-bd65-922da6a1a824 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:04.135809 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:04.135809 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:04.135809 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:04.135809 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:04.135809 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:04.135809 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:04.135809 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:04.135809 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:04.135809 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:04.135809 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:04.135809 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:04.135809 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:04.135809 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:04.135809 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:04.136847 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:04.136847 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:04.136847 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:04.136847 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:04.136847 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:04.136847 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:04.136847 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:04.136847 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:04.136847 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:04.136847 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:04.136847 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:04.136847 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:04.136847 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:04.136847 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:04.136847 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:04.136847 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:04.136847 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:04.137659 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:04.137659 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:04.137659 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:04.137659 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:04.137861 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b0df9-2b0b-4258-9ce1-8970c03494cb None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:04.141700 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 254/496] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:04.142551 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c427f17-7f33-438a-bd65-922da6a1a824 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:04.178383 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 243/497] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Aug 1 13:56:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:04.178383 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b0df9-2b0b-4258-9ce1-8970c03494cb None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:04.201111 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:04.251099 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c427f17-7f33-438a-bd65-922da6a1a824 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:04.286874 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b0df9-2b0b-4258-9ce1-8970c03494cb None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:04.290068 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb126de-cbb9-4c9f-bd87-a49c8d0d7ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:04.290725 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb126de-cbb9-4c9f-bd87-a49c8d0d7ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:04.291013 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb126de-cbb9-4c9f-bd87-a49c8d0d7ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:04.291296 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb126de-cbb9-4c9f-bd87-a49c8d0d7ea7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:04.304041 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb126de-cbb9-4c9f-bd87-a49c8d0d7ea7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:56:04.310772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-5bb126de-cbb9-4c9f-bd87-a49c8d0d7ea7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:56:04.310772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:04.310772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:04.310772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:04.310772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:04.310772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:04.310772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:04.310772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:04.310772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:04.310772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:04.310772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:04.310772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:04.310772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:56:04.310772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:56:04.310772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:56:04.310772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:56:04.311869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:56:04.311869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:56:04.311869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:56:04.311869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:56:04.311869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:04.311869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:04.311869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:04.311869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:04.311869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:04.311869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:04.311869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:04.311869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:04.311869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:04.311869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:04.311869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:04.311869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:04.312687 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:04.312687 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:04.312687 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:04.312687 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:04.312687 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:56:04.312687 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:56:04.312687 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:56:04.312687 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:56:04.312687 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:56:04.312687 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:56:04.312687 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:56:04.312687 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:04.316632 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 255/498] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 13:56:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:04.317789 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb126de-cbb9-4c9f-bd87-a49c8d0d7ea7 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:04.322004 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:04.327307 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cffcfb5-6877-461b-ada3-da988abf0481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:04.327896 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cffcfb5-6877-461b-ada3-da988abf0481 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:04.328182 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cffcfb5-6877-461b-ada3-da988abf0481 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:04.328459 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cffcfb5-6877-461b-ada3-da988abf0481 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:04.339954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cffcfb5-6877-461b-ada3-da988abf0481 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:56:04.344911 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-0cffcfb5-6877-461b-ada3-da988abf0481 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:56:04.344911 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:04.344911 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:04.344911 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:04.344911 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:04.344911 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:04.344911 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:04.344911 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:04.344911 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:04.344911 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:04.344911 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:04.344911 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:04.344911 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:56:04.344911 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:56:04.344911 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:56:04.344911 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:56:04.345781 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:56:04.345781 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:56:04.345781 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:56:04.345781 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:56:04.345781 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:04.345781 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:04.345781 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:04.345781 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:04.345781 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:04.345781 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:04.345781 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:04.345781 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:04.345781 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:04.345781 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:04.345781 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:04.345781 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:04.351910 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:04.351910 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:04.351910 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:04.351910 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:04.351910 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:56:04.351910 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:56:04.351910 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:56:04.351910 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:56:04.351910 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:56:04.351910 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:56:04.351910 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:56:04.351910 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:04.351910 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 244/499] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 13:56:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:04.351910 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cffcfb5-6877-461b-ada3-da988abf0481 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:04.428101 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb126de-cbb9-4c9f-bd87-a49c8d0d7ea7 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:04.441307 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:04.460495 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cffcfb5-6877-461b-ada3-da988abf0481 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:04.462664 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d6b3d-20ee-4e9f-b4fe-25b22f88190e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:04.463286 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d6b3d-20ee-4e9f-b4fe-25b22f88190e None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:04.463569 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d6b3d-20ee-4e9f-b4fe-25b22f88190e None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:04.463850 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d6b3d-20ee-4e9f-b4fe-25b22f88190e None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:04.464574 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d6b3d-20ee-4e9f-b4fe-25b22f88190e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:04.464887 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d6b3d-20ee-4e9f-b4fe-25b22f88190e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:04.466269 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d6b3d-20ee-4e9f-b4fe-25b22f88190e None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:04.478606 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 256/500] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:04.479445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d6b3d-20ee-4e9f-b4fe-25b22f88190e None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:04.495498 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d7f0c-773a-4340-aa9b-0be3f2a34a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:04.496106 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d7f0c-773a-4340-aa9b-0be3f2a34a72 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:04.496385 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d7f0c-773a-4340-aa9b-0be3f2a34a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:04.496696 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d7f0c-773a-4340-aa9b-0be3f2a34a72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:04.546762 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:04.566920 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98d7f0c-773a-4340-aa9b-0be3f2a34a72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:04.568386 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98d7f0c-773a-4340-aa9b-0be3f2a34a72 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:04.584203 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d6b3d-20ee-4e9f-b4fe-25b22f88190e None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:04.618170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e842ced-5196-44a7-b455-d8da3f0dc0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:04.618781 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e842ced-5196-44a7-b455-d8da3f0dc0f4 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:04.619064 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e842ced-5196-44a7-b455-d8da3f0dc0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:04.619343 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e842ced-5196-44a7-b455-d8da3f0dc0f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:04.620057 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e842ced-5196-44a7-b455-d8da3f0dc0f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:04.620369 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e842ced-5196-44a7-b455-d8da3f0dc0f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:04.621769 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e842ced-5196-44a7-b455-d8da3f0dc0f4 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:04.634489 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 257/501] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 13:56:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:04.634489 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e842ced-5196-44a7-b455-d8da3f0dc0f4 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:04.651319 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:04.671807 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 245/502] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:56:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:56:04.673258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099728ce-3735-44fd-beee-a9bbeb2b4047 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:04.673545 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099728ce-3735-44fd-beee-a9bbeb2b4047 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:04.673822 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099728ce-3735-44fd-beee-a9bbeb2b4047 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:04.675059 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 246/503] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:04.675751 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099728ce-3735-44fd-beee-a9bbeb2b4047 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:04.733563 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e842ced-5196-44a7-b455-d8da3f0dc0f4 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:04.758412 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:04.767681 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c717e07-44f3-4670-b3f4-32051deaf4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:04.768354 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c717e07-44f3-4670-b3f4-32051deaf4bd None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:04.768674 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c717e07-44f3-4670-b3f4-32051deaf4bd None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:04.768952 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c717e07-44f3-4670-b3f4-32051deaf4bd None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:04.769670 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c717e07-44f3-4670-b3f4-32051deaf4bd None admin] RBAC: Adding query filter params (name=service) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:04.769974 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c717e07-44f3-4670-b3f4-32051deaf4bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:04.771332 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c717e07-44f3-4670-b3f4-32051deaf4bd None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:04.777032 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099728ce-3735-44fd-beee-a9bbeb2b4047 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:04.779664 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 258/504] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 13:56:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:04.780439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c717e07-44f3-4670-b3f4-32051deaf4bd None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:04.810766 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5591790-452c-40ac-b95f-79feb9c2459a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:04.811412 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5591790-452c-40ac-b95f-79feb9c2459a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:04.812145 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5591790-452c-40ac-b95f-79feb9c2459a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:04.812420 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5591790-452c-40ac-b95f-79feb9c2459a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:04.819970 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5591790-452c-40ac-b95f-79feb9c2459a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:04.822003 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5591790-452c-40ac-b95f-79feb9c2459a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:04.833448 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-e5591790-452c-40ac-b95f-79feb9c2459a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 13:56:04.833448 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:04.833448 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:04.833448 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:04.833448 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:04.833448 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:04.833448 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:04.833448 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:04.833448 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:04.833448 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:04.833448 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:04.833448 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:04.833448 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:56:04.833448 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:56:04.833448 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:56:04.833448 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:56:04.834349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:04.834349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:04.834349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:56:04.834349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:04.834349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:56:04.834349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:04.834349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:04.834349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:04.834349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:04.834349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:04.834349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:04.834349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:04.834349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:04.834349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:04.834349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:04.834349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:04.834349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:04.834349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:04.835261 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:04.835261 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:04.835261 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:56:04.835261 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:56:04.835261 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:56:04.835261 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:56:04.835261 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:56:04.835261 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:56:04.835261 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 13:56:04.835261 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:04.835261 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 247/505] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:56:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:04.835811 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5591790-452c-40ac-b95f-79feb9c2459a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:04.882450 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:04.886266 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c717e07-44f3-4670-b3f4-32051deaf4bd None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:04.919951 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2953fc1-adb8-4f3e-bb11-8b13c6118f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:04.920635 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2953fc1-adb8-4f3e-bb11-8b13c6118f0d None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:04.920922 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2953fc1-adb8-4f3e-bb11-8b13c6118f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:04.921203 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2953fc1-adb8-4f3e-bb11-8b13c6118f0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:04.931922 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2953fc1-adb8-4f3e-bb11-8b13c6118f0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:04.933903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2953fc1-adb8-4f3e-bb11-8b13c6118f0d None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:04.942029 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5591790-452c-40ac-b95f-79feb9c2459a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:04.947958 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-e2953fc1-adb8-4f3e-bb11-8b13c6118f0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:04.947958 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:04.947958 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:04.947958 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:04.947958 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:04.947958 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:04.947958 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:04.947958 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:04.947958 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:04.947958 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:04.947958 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:04.947958 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:04.947958 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:04.947958 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:04.947958 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:04.949009 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:04.949009 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:04.949009 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:04.949009 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:04.949009 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:04.949009 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:04.949009 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:04.949009 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:04.949009 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:04.949009 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:04.949009 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:04.949009 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:04.949009 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:04.949009 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:04.949009 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:04.949009 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:04.949009 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:04.949871 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:04.949871 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:04.949871 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:04.949871 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:04.949871 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 259/506] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:04.950659 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42ff8b-3a24-4466-b74e-05faa7c7aeef None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:04.950944 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42ff8b-3a24-4466-b74e-05faa7c7aeef None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:04.951217 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42ff8b-3a24-4466-b74e-05faa7c7aeef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:04.968156 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-fe42ff8b-3a24-4466-b74e-05faa7c7aeef None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:04.969354 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-fe42ff8b-3a24-4466-b74e-05faa7c7aeef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:04.978468 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c143ab-815d-474f-a17f-c6249a0e5fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:04.979246 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c143ab-815d-474f-a17f-c6249a0e5fc3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:04.979532 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c143ab-815d-474f-a17f-c6249a0e5fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:04.979811 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c143ab-815d-474f-a17f-c6249a0e5fc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:04.990396 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c143ab-815d-474f-a17f-c6249a0e5fc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:04.991790 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c143ab-815d-474f-a17f-c6249a0e5fc3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:05.003388 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:05.006299 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-b1c143ab-815d-474f-a17f-c6249a0e5fc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:05.006299 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:05.006299 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:05.006299 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:05.006299 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:05.006299 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:05.006299 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:05.006299 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:05.006299 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:05.006299 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:05.006299 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:05.006299 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:05.006299 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:05.006299 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:05.006299 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:05.007360 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:05.007360 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:05.007360 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:05.007360 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:05.007360 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:05.007360 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:05.007360 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:05.007360 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:05.007360 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:05.007360 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:05.007360 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:05.007360 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:05.007360 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:05.007360 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:05.007360 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:05.007360 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:05.007360 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:05.008286 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:05.008286 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:05.008286 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:05.008286 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:05.008286 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 248/507] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:05.009018 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c143ab-815d-474f-a17f-c6249a0e5fc3 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:05.037351 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 260/508] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:56:05.038196 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42ff8b-3a24-4466-b74e-05faa7c7aeef None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:05.103091 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:05.111974 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c143ab-815d-474f-a17f-c6249a0e5fc3 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:05.146348 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b260de51-852a-48bd-ab84-7e8a5b298e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:05.146998 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b260de51-852a-48bd-ab84-7e8a5b298e8a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:05.147366 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b260de51-852a-48bd-ab84-7e8a5b298e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:05.147651 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b260de51-852a-48bd-ab84-7e8a5b298e8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:05.147804 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42ff8b-3a24-4466-b74e-05faa7c7aeef None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:05.159067 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b260de51-852a-48bd-ab84-7e8a5b298e8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:05.160563 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b260de51-852a-48bd-ab84-7e8a5b298e8a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:05.175220 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-b260de51-852a-48bd-ab84-7e8a5b298e8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:05.175220 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:05.175220 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:05.175220 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:05.175220 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:05.175220 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:05.175220 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:05.175220 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:05.175220 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:05.175220 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:05.175220 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:05.175220 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:05.175220 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:05.175220 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:05.175220 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:05.176067 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:05.176067 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:05.176067 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:05.176067 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:05.176067 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:05.176067 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:05.176067 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:05.176067 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:05.176067 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:05.176067 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:05.176067 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:05.176067 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:05.176067 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:05.176067 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:05.176067 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:05.176067 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:05.176067 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:05.176938 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:05.176938 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:05.176938 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:05.176938 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:05.176938 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 249/509] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:05.177302 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b260de51-852a-48bd-ab84-7e8a5b298e8a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:05.187683 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95bccd5-27c9-4eba-8c7d-0a8b10120b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:05.188307 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95bccd5-27c9-4eba-8c7d-0a8b10120b1b None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:05.188626 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95bccd5-27c9-4eba-8c7d-0a8b10120b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:05.188905 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95bccd5-27c9-4eba-8c7d-0a8b10120b1b None admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:05.189706 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95bccd5-27c9-4eba-8c7d-0a8b10120b1b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:05.190016 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95bccd5-27c9-4eba-8c7d-0a8b10120b1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:05.191448 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95bccd5-27c9-4eba-8c7d-0a8b10120b1b None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:05.228354 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 261/510] 10.222.0.48 () {56 vars in 1137 bytes} [Mon Aug 1 13:56:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:05.229198 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95bccd5-27c9-4eba-8c7d-0a8b10120b1b None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:05.256153 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:05.280049 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b260de51-852a-48bd-ab84-7e8a5b298e8a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:05.312686 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc3a005-4807-4eee-a3e4-4567831d82ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:05.313295 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3a005-4807-4eee-a3e4-4567831d82ef None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:05.313572 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3a005-4807-4eee-a3e4-4567831d82ef None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:05.313843 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3a005-4807-4eee-a3e4-4567831d82ef None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:05.314527 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc3a005-4807-4eee-a3e4-4567831d82ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:05.314828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc3a005-4807-4eee-a3e4-4567831d82ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:05.316181 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc3a005-4807-4eee-a3e4-4567831d82ef None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:05.327551 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 250/511] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:05.327878 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95bccd5-27c9-4eba-8c7d-0a8b10120b1b None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:05.328391 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc3a005-4807-4eee-a3e4-4567831d82ef None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:05.332730 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:05.363101 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a951bd4-9e82-4f96-9d8f-02d931fbd69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:05.363710 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a951bd4-9e82-4f96-9d8f-02d931fbd69b None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:05.363998 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a951bd4-9e82-4f96-9d8f-02d931fbd69b None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:05.364271 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a951bd4-9e82-4f96-9d8f-02d931fbd69b None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:05.364996 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a951bd4-9e82-4f96-9d8f-02d931fbd69b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:05.365296 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a951bd4-9e82-4f96-9d8f-02d931fbd69b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:05.366668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a951bd4-9e82-4f96-9d8f-02d931fbd69b None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:05.377599 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 262/512] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:05.378377 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a951bd4-9e82-4f96-9d8f-02d931fbd69b None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:05.429224 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc3a005-4807-4eee-a3e4-4567831d82ef None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:05.463363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91798223-0109-4d2a-b57c-6337093955d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:05.464005 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91798223-0109-4d2a-b57c-6337093955d6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:05.464292 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91798223-0109-4d2a-b57c-6337093955d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:05.464598 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91798223-0109-4d2a-b57c-6337093955d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:05.474474 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91798223-0109-4d2a-b57c-6337093955d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:05.474659 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:05.475933 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91798223-0109-4d2a-b57c-6337093955d6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:05.487800 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a951bd4-9e82-4f96-9d8f-02d931fbd69b None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:05.491026 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-91798223-0109-4d2a-b57c-6337093955d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:05.491026 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:05.491026 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:05.491026 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:05.491026 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:05.491026 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:05.491026 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:05.491026 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:05.491026 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:05.491026 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:05.491026 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:05.491026 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:05.491026 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:05.491026 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:05.491026 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:05.491966 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:05.491966 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:05.491966 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:05.491966 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:05.491966 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:05.491966 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:05.491966 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:05.491966 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:05.491966 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:05.491966 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:05.491966 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:05.491966 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:05.491966 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:05.491966 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:05.491966 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:05.491966 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:05.491966 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:05.493080 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:05.493080 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:05.493080 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:05.493080 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:05.496646 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 251/513] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:05.498062 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91798223-0109-4d2a-b57c-6337093955d6 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:05.522407 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad998e7-77d5-4163-ac45-f37635cab330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:05.523033 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad998e7-77d5-4163-ac45-f37635cab330 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:05.523235 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad998e7-77d5-4163-ac45-f37635cab330 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:05.523509 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad998e7-77d5-4163-ac45-f37635cab330 None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:05.524939 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad998e7-77d5-4163-ac45-f37635cab330 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:05.525257 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad998e7-77d5-4163-ac45-f37635cab330 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:05.526607 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad998e7-77d5-4163-ac45-f37635cab330 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:05.538042 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 263/514] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:05.538636 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad998e7-77d5-4163-ac45-f37635cab330 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:05.554698 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:05.598233 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91798223-0109-4d2a-b57c-6337093955d6 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:05.631301 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c751e-bff7-4ef2-ae2e-3163ccbdc36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:05.632182 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c751e-bff7-4ef2-ae2e-3163ccbdc36e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:05.632464 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c751e-bff7-4ef2-ae2e-3163ccbdc36e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:05.632776 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c751e-bff7-4ef2-ae2e-3163ccbdc36e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:05.638131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad998e7-77d5-4163-ac45-f37635cab330 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:05.671816 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46401c7-852d-400a-a8b7-37ae1a057249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:05.672447 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46401c7-852d-400a-a8b7-37ae1a057249 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:05.672755 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46401c7-852d-400a-a8b7-37ae1a057249 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:05.673029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46401c7-852d-400a-a8b7-37ae1a057249 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:05.679970 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46401c7-852d-400a-a8b7-37ae1a057249 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:05.681486 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46401c7-852d-400a-a8b7-37ae1a057249 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:05.694192 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-d46401c7-852d-400a-a8b7-37ae1a057249 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 13:56:05.694192 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:05.694192 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:05.694192 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:05.694192 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:05.694192 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:05.694192 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:05.694192 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:05.694192 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:05.694192 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:05.694192 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:05.694192 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:05.694192 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:56:05.694192 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:56:05.694192 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:56:05.694192 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:56:05.695079 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:05.695079 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:05.695079 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:56:05.695079 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:05.695079 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:56:05.695079 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:05.695079 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:05.695079 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:05.695079 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:05.695079 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:05.695079 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:05.695079 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:05.695079 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:05.695079 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:05.695079 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:05.695079 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:05.695079 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:05.695079 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:05.695965 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:05.695965 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:05.695965 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:56:05.695965 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:56:05.695965 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:56:05.695965 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:56:05.695965 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:56:05.695965 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:56:05.695965 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 13:56:05.695965 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:05.695965 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 264/515] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 13:56:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:05.697079 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46401c7-852d-400a-a8b7-37ae1a057249 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:05.704143 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c751e-bff7-4ef2-ae2e-3163ccbdc36e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:05.705652 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c751e-bff7-4ef2-ae2e-3163ccbdc36e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:05.711508 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:05.760415 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:05.800502 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46401c7-852d-400a-a8b7-37ae1a057249 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:05.807517 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 252/516] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:56:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:56:05.808820 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c751e-bff7-4ef2-ae2e-3163ccbdc36e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:05.834278 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66fb75-c5ba-46c0-b3f7-e855bf2c2cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:05.834920 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66fb75-c5ba-46c0-b3f7-e855bf2c2cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:05.835203 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66fb75-c5ba-46c0-b3f7-e855bf2c2cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:05.835477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66fb75-c5ba-46c0-b3f7-e855bf2c2cb0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:05.842042 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b66fb75-c5ba-46c0-b3f7-e855bf2c2cb0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:05.843500 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b66fb75-c5ba-46c0-b3f7-e855bf2c2cb0 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:05.855698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-9b66fb75-c5ba-46c0-b3f7-e855bf2c2cb0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:05.855698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:05.855698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:05.855698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:05.855698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:05.855698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:05.855698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:05.855698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:05.855698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:05.855698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:05.855698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:05.855698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:05.855698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:56:05.855698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:56:05.855698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:56:05.856538 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:56:05.856538 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:05.856538 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:05.856538 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:05.856538 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:05.856538 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:05.856538 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:05.856538 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:05.856538 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:05.856538 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:05.856538 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:05.856538 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:05.856538 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:05.856538 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:05.856538 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:05.856538 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:05.856538 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:05.858557 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:56:05.858557 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:56:05.858557 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:56:05.858557 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:56:05.858557 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:56:05.858557 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:56:05.858557 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:05.858557 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:05.858557 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 265/517] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:56:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:05.858557 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66fb75-c5ba-46c0-b3f7-e855bf2c2cb0 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:05.916140 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c751e-bff7-4ef2-ae2e-3163ccbdc36e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:05.925382 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:05.950745 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d4cdd-b157-4510-b6b9-4d3976d7bd64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:05.951367 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d4cdd-b157-4510-b6b9-4d3976d7bd64 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:05.951651 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d4cdd-b157-4510-b6b9-4d3976d7bd64 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:05.951930 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d4cdd-b157-4510-b6b9-4d3976d7bd64 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:05.952668 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726d4cdd-b157-4510-b6b9-4d3976d7bd64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:05.953030 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726d4cdd-b157-4510-b6b9-4d3976d7bd64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:05.954409 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726d4cdd-b157-4510-b6b9-4d3976d7bd64 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:05.960935 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66fb75-c5ba-46c0-b3f7-e855bf2c2cb0 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:05.966085 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 253/518] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:05.966714 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d4cdd-b157-4510-b6b9-4d3976d7bd64 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:05.988064 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:05.995110 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791166a4-4d1d-4874-978d-610d7500f331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:05.995756 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791166a4-4d1d-4874-978d-610d7500f331 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:05.996042 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791166a4-4d1d-4874-978d-610d7500f331 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:05.996324 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791166a4-4d1d-4874-978d-610d7500f331 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:06.004222 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791166a4-4d1d-4874-978d-610d7500f331 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:06.005705 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791166a4-4d1d-4874-978d-610d7500f331 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:06.017008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-791166a4-4d1d-4874-978d-610d7500f331 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 13:56:06.017008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:06.017008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:06.017008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:06.017008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:06.017008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:06.017008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:06.017008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:06.017008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:06.017008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:06.017008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:06.017008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:06.017008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:56:06.017008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:56:06.017008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:56:06.017008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:56:06.018050 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:06.018050 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:06.018050 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:56:06.018050 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:06.018050 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:56:06.018050 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:06.018050 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:06.018050 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:06.018050 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:06.018050 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:06.018050 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:06.018050 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:06.018050 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:06.018050 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:06.018050 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:06.018050 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:06.018050 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:06.018050 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:06.023631 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:06.023631 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:06.023631 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:56:06.023631 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:56:06.023631 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:56:06.023631 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:56:06.023631 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:56:06.023631 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:56:06.023631 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 13:56:06.023631 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:06.023631 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 266/519] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:56:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:06.023631 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791166a4-4d1d-4874-978d-610d7500f331 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:06.069300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d4cdd-b157-4510-b6b9-4d3976d7bd64 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:06.101604 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531394e-2998-4cc4-898f-c82863483d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:06.102305 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531394e-2998-4cc4-898f-c82863483d29 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:06.102592 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531394e-2998-4cc4-898f-c82863483d29 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:06.102874 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531394e-2998-4cc4-898f-c82863483d29 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:06.103670 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531394e-2998-4cc4-898f-c82863483d29 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:06.104037 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531394e-2998-4cc4-898f-c82863483d29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:06.105489 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531394e-2998-4cc4-898f-c82863483d29 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:06.116570 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791166a4-4d1d-4874-978d-610d7500f331 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:06.122525 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 254/520] 10.222.0.48 () {56 vars in 1139 bytes} [Mon Aug 1 13:56:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:06.123326 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531394e-2998-4cc4-898f-c82863483d29 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:06.147001 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:06.155134 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a5cc4-e945-476f-a83d-74e0ccb962b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:06.155844 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a5cc4-e945-476f-a83d-74e0ccb962b8 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:06.156192 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a5cc4-e945-476f-a83d-74e0ccb962b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:06.156563 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a5cc4-e945-476f-a83d-74e0ccb962b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:06.166888 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2a5cc4-e945-476f-a83d-74e0ccb962b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:06.168317 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2a5cc4-e945-476f-a83d-74e0ccb962b8 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:06.182795 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-4f2a5cc4-e945-476f-a83d-74e0ccb962b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:06.182795 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:06.182795 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:06.182795 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:06.182795 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:06.182795 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:06.182795 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:06.182795 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:06.182795 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:06.182795 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:06.182795 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:06.182795 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:06.182795 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:06.182795 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:06.182795 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:06.183791 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:06.183791 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:06.183791 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:06.183791 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:06.183791 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:06.183791 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:06.183791 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:06.183791 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:06.183791 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:06.183791 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:06.183791 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:06.183791 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:06.183791 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:06.183791 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:06.183791 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:06.183791 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:06.183791 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:06.184889 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:06.184889 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:06.184889 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:06.184889 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:06.184889 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 267/521] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:06.184889 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a5cc4-e945-476f-a83d-74e0ccb962b8 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:06.199398 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:06.224285 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531394e-2998-4cc4-898f-c82863483d29 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:06.257464 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed907d4-cdd7-4020-8c0e-766394515154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:06.258075 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed907d4-cdd7-4020-8c0e-766394515154 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:06.258364 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed907d4-cdd7-4020-8c0e-766394515154 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:06.259229 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed907d4-cdd7-4020-8c0e-766394515154 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:06.259980 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed907d4-cdd7-4020-8c0e-766394515154 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:06.260296 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed907d4-cdd7-4020-8c0e-766394515154 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:06.261750 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed907d4-cdd7-4020-8c0e-766394515154 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:06.278767 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 255/522] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 13:56:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:06.279568 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed907d4-cdd7-4020-8c0e-766394515154 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:06.282481 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a5cc4-e945-476f-a83d-74e0ccb962b8 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:06.317339 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34081e19-46ed-4233-b6eb-590cae158b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:06.317963 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34081e19-46ed-4233-b6eb-590cae158b26 None admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:06.318246 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34081e19-46ed-4233-b6eb-590cae158b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:06.318519 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34081e19-46ed-4233-b6eb-590cae158b26 None admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:06.319189 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34081e19-46ed-4233-b6eb-590cae158b26 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:06.320740 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34081e19-46ed-4233-b6eb-590cae158b26 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:06.340792 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 268/523] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 13:56:06 2022] POST /identity/v3/users => generated 247 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:56:06.341655 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34081e19-46ed-4233-b6eb-590cae158b26 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:06.380978 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:06.390352 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed907d4-cdd7-4020-8c0e-766394515154 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:06.429751 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a1ed98-ece9-4cee-9ca3-3bac2f836782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:06.431466 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1ed98-ece9-4cee-9ca3-3bac2f836782 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:06.432032 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1ed98-ece9-4cee-9ca3-3bac2f836782 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:06.432378 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1ed98-ece9-4cee-9ca3-3bac2f836782 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:06.433287 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a1ed98-ece9-4cee-9ca3-3bac2f836782 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:06.433691 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a1ed98-ece9-4cee-9ca3-3bac2f836782 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:06.435301 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a1ed98-ece9-4cee-9ca3-3bac2f836782 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:06.438021 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:06.461915 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 256/524] 10.222.0.48 () {56 vars in 1137 bytes} [Mon Aug 1 13:56:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:06.461915 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a1ed98-ece9-4cee-9ca3-3bac2f836782 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:06.462237 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34081e19-46ed-4233-b6eb-590cae158b26 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:06.500501 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdbe95a-b445-4c6d-abe1-d9649de16582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:06.501134 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdbe95a-b445-4c6d-abe1-d9649de16582 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:06.501419 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdbe95a-b445-4c6d-abe1-d9649de16582 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:06.501701 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdbe95a-b445-4c6d-abe1-d9649de16582 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:06.580778 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdbe95a-b445-4c6d-abe1-d9649de16582 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:06.582158 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdbe95a-b445-4c6d-abe1-d9649de16582 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:06.583733 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a1ed98-ece9-4cee-9ca3-3bac2f836782 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:06.620261 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:06.622358 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37549d08-86dd-49fb-b29b-7a142dc1f6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:06.623100 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37549d08-86dd-49fb-b29b-7a142dc1f6f5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:06.623482 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37549d08-86dd-49fb-b29b-7a142dc1f6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:06.623825 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37549d08-86dd-49fb-b29b-7a142dc1f6f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:06.624622 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37549d08-86dd-49fb-b29b-7a142dc1f6f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:06.625016 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37549d08-86dd-49fb-b29b-7a142dc1f6f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:06.626477 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37549d08-86dd-49fb-b29b-7a142dc1f6f5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:06.643090 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 257/525] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:06.643090 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37549d08-86dd-49fb-b29b-7a142dc1f6f5 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:06.694430 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 269/526] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:56:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:56:06.695879 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdbe95a-b445-4c6d-abe1-d9649de16582 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:06.705951 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:06.765198 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37549d08-86dd-49fb-b29b-7a142dc1f6f5 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:06.801009 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75337f38-b40a-48c2-ad16-fea7f3b62f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:06.801637 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75337f38-b40a-48c2-ad16-fea7f3b62f7c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:06.801922 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75337f38-b40a-48c2-ad16-fea7f3b62f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:06.802204 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75337f38-b40a-48c2-ad16-fea7f3b62f7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:06.803001 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75337f38-b40a-48c2-ad16-fea7f3b62f7c None admin] RBAC: Adding query filter params (role.id=d3d8169e71454005a8cf0d7544ad05dc, scope.project.id=72ecec9fad964b1fbeaefd0519597229, user.id=a246654c783b4cc68d212e91bb05d8c6) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:06.803433 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75337f38-b40a-48c2-ad16-fea7f3b62f7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:06.805016 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75337f38-b40a-48c2-ad16-fea7f3b62f7c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:06.811305 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 258/527] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:56:06 2022] GET /identity/v3/role_assignments?user.id=a246654c783b4cc68d212e91bb05d8c6&scope.project.id=72ecec9fad964b1fbeaefd0519597229&role.id=d3d8169e71454005a8cf0d7544ad05dc => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:06.813115 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdbe95a-b445-4c6d-abe1-d9649de16582 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:06.825546 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75337f38-b40a-48c2-ad16-fea7f3b62f7c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:06.864156 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a5311-f5d9-45f0-a560-ae06fa4d366b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:06.864785 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a5311-f5d9-45f0-a560-ae06fa4d366b None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:06.865077 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a5311-f5d9-45f0-a560-ae06fa4d366b None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:06.865357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a5311-f5d9-45f0-a560-ae06fa4d366b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:06.871218 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:06.957380 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824a5311-f5d9-45f0-a560-ae06fa4d366b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:06.959246 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824a5311-f5d9-45f0-a560-ae06fa4d366b None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:06.970896 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75337f38-b40a-48c2-ad16-fea7f3b62f7c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:06.989225 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:07.019491 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6520be0-76cf-49e3-a12d-9204ddf76952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:07.020555 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6520be0-76cf-49e3-a12d-9204ddf76952 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:07.021033 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6520be0-76cf-49e3-a12d-9204ddf76952 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:07.021440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6520be0-76cf-49e3-a12d-9204ddf76952 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:07.040615 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6520be0-76cf-49e3-a12d-9204ddf76952 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:07.042624 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6520be0-76cf-49e3-a12d-9204ddf76952 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:07.056036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-b6520be0-76cf-49e3-a12d-9204ddf76952 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:07.056036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:07.056036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:07.056036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:07.056036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:07.056036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:07.056036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:07.056036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:07.056036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:07.056036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:07.056036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:07.056036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:07.056036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:56:07.056036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:56:07.056036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:56:07.057125 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:56:07.057125 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:07.057125 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:07.057125 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:07.057125 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:07.057125 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:07.057125 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:07.057125 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:07.057125 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:07.057125 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:07.057125 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:07.057125 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:07.057125 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:07.057125 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:07.057125 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:07.057125 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:07.057125 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:07.058025 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:56:07.058025 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:56:07.058025 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:56:07.058025 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:56:07.058025 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:56:07.058025 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:56:07.058025 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:07.058025 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:07.060684 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 259/528] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:56:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:07.061983 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6520be0-76cf-49e3-a12d-9204ddf76952 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:07.081305 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 270/529] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:56:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:56:07.081305 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a5311-f5d9-45f0-a560-ae06fa4d366b None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:07.122411 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:07.196748 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6520be0-76cf-49e3-a12d-9204ddf76952 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:07.200768 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a5311-f5d9-45f0-a560-ae06fa4d366b None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:07.241129 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627e207-ce66-4ac4-96b6-302843d84ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:07.242558 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627e207-ce66-4ac4-96b6-302843d84ecf None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:07.243108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e1064-f108-462c-8f19-cec4c0e38459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:07.243683 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627e207-ce66-4ac4-96b6-302843d84ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:07.243802 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e1064-f108-462c-8f19-cec4c0e38459 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:07.244086 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e1064-f108-462c-8f19-cec4c0e38459 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:07.244376 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e1064-f108-462c-8f19-cec4c0e38459 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:07.244840 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627e207-ce66-4ac4-96b6-302843d84ecf None admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:07.245845 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7627e207-ce66-4ac4-96b6-302843d84ecf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:07.246446 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7627e207-ce66-4ac4-96b6-302843d84ecf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:07.248195 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7627e207-ce66-4ac4-96b6-302843d84ecf None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:07.256154 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212e1064-f108-462c-8f19-cec4c0e38459 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:07.257900 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212e1064-f108-462c-8f19-cec4c0e38459 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:07.269791 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 271/530] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 13:56:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:07.270976 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627e207-ce66-4ac4-96b6-302843d84ecf None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:07.276593 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-212e1064-f108-462c-8f19-cec4c0e38459 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:07.276593 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:07.276593 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:07.276593 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:07.276593 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:07.276593 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:07.276593 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:07.276593 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:07.276593 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:07.276593 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:07.276593 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:07.276593 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:07.276593 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:07.276593 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:07.276593 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:07.277456 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:07.277456 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:07.277456 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:07.277456 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:07.277456 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:07.277456 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:07.277456 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:07.277456 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:07.277456 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:07.277456 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:07.277456 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:07.277456 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:07.277456 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:07.277456 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:07.277456 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:07.277456 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:07.277456 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:07.278266 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:07.278266 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:07.278266 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:07.278266 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:07.278266 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 260/531] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:07.278951 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e1064-f108-462c-8f19-cec4c0e38459 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:07.315903 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:07.397952 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:07.403581 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e1064-f108-462c-8f19-cec4c0e38459 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:07.406927 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627e207-ce66-4ac4-96b6-302843d84ecf None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:07.439928 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03296788-508e-47ca-aa07-baf8da461aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:07.444985 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03296788-508e-47ca-aa07-baf8da461aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:07.444985 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03296788-508e-47ca-aa07-baf8da461aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:07.444985 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03296788-508e-47ca-aa07-baf8da461aa5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:07.444985 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03296788-508e-47ca-aa07-baf8da461aa5 None admin] RBAC: Adding query filter params (role.id=d3d8169e71454005a8cf0d7544ad05dc, scope.project.id=72ecec9fad964b1fbeaefd0519597229, user.id=b185b052f9df4d3f9847d8f9b4cfda5f) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:07.444985 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03296788-508e-47ca-aa07-baf8da461aa5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:07.444985 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03296788-508e-47ca-aa07-baf8da461aa5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:07.452393 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 261/532] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:56:07 2022] GET /identity/v3/role_assignments?user.id=b185b052f9df4d3f9847d8f9b4cfda5f&scope.project.id=72ecec9fad964b1fbeaefd0519597229&role.id=d3d8169e71454005a8cf0d7544ad05dc => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:07.453474 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03296788-508e-47ca-aa07-baf8da461aa5 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:07.455212 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feecb590-b6df-4f9e-a565-4c984a8c2937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:07.456054 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feecb590-b6df-4f9e-a565-4c984a8c2937 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:07.456801 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feecb590-b6df-4f9e-a565-4c984a8c2937 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:07.457255 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feecb590-b6df-4f9e-a565-4c984a8c2937 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:07.483705 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feecb590-b6df-4f9e-a565-4c984a8c2937 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:07.485649 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feecb590-b6df-4f9e-a565-4c984a8c2937 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:07.522737 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-feecb590-b6df-4f9e-a565-4c984a8c2937 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:07.522737 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:07.522737 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:07.522737 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:07.522737 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:07.522737 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:07.522737 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:07.522737 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:07.522737 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:07.522737 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:07.522737 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:07.522737 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:07.522737 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:07.522737 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:07.522737 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:07.523822 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:07.523822 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:07.523822 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:07.523822 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:07.523822 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:07.523822 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:07.523822 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:07.523822 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:07.523822 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:07.523822 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:07.523822 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:07.523822 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:07.523822 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:07.523822 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:07.523822 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:07.523822 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:07.523822 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:07.524949 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:07.524949 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:07.524949 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:07.524949 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:07.527789 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 272/533] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:07.536260 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feecb590-b6df-4f9e-a565-4c984a8c2937 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:07.627835 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:07.640742 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03296788-508e-47ca-aa07-baf8da461aa5 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:07.679149 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64161eb3-081f-4473-a05c-3ad6578ee668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:07.680057 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64161eb3-081f-4473-a05c-3ad6578ee668 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:07.681122 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64161eb3-081f-4473-a05c-3ad6578ee668 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:07.681554 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64161eb3-081f-4473-a05c-3ad6578ee668 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:07.682397 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64161eb3-081f-4473-a05c-3ad6578ee668 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:07.682850 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64161eb3-081f-4473-a05c-3ad6578ee668 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:07.685082 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64161eb3-081f-4473-a05c-3ad6578ee668 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:07.697923 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feecb590-b6df-4f9e-a565-4c984a8c2937 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:07.700193 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 262/534] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:07.707022 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64161eb3-081f-4473-a05c-3ad6578ee668 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:07.743557 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2731595-f3b2-49f9-8d8f-e841b3f297ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:07.744179 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2731595-f3b2-49f9-8d8f-e841b3f297ff None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:07.744461 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2731595-f3b2-49f9-8d8f-e841b3f297ff None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:07.744761 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2731595-f3b2-49f9-8d8f-e841b3f297ff None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:07.745457 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2731595-f3b2-49f9-8d8f-e841b3f297ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:07.745762 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2731595-f3b2-49f9-8d8f-e841b3f297ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:07.747130 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2731595-f3b2-49f9-8d8f-e841b3f297ff None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:07.774954 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 273/535] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:07.775839 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2731595-f3b2-49f9-8d8f-e841b3f297ff None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:07.792994 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:07.899998 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64161eb3-081f-4473-a05c-3ad6578ee668 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:07.945081 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2731595-f3b2-49f9-8d8f-e841b3f297ff None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:07.981081 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c0f8a-f685-45e1-a8ff-0f489b03f4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:07.981750 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c0f8a-f685-45e1-a8ff-0f489b03f4db None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:07.982031 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c0f8a-f685-45e1-a8ff-0f489b03f4db None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:07.982306 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c0f8a-f685-45e1-a8ff-0f489b03f4db None admin] PATH_INFO: `/v3/projects/service` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:07.990319 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6c0f8a-f685-45e1-a8ff-0f489b03f4db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:07.991786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6c0f8a-f685-45e1-a8ff-0f489b03f4db None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:08.004992 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-5d6c0f8a-f685-45e1-a8ff-0f489b03f4db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:08.004992 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:08.004992 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:08.004992 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:08.004992 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:08.004992 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:08.004992 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:08.004992 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:08.004992 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:08.004992 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:08.004992 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:08.004992 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:08.004992 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:56:08.004992 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:56:08.004992 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:56:08.005894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:56:08.005894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:08.005894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:08.005894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:08.005894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:08.005894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:08.005894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:08.005894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:08.005894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:08.005894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:08.005894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:08.005894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:08.005894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:08.005894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:08.005894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:08.005894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:08.005894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:08.006957 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:56:08.006957 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:56:08.006957 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:56:08.006957 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:56:08.006957 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:56:08.006957 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:56:08.006957 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:08.006957 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:08.007360 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80804b2-503c-45b2-9db6-1bcc9538d080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:08.007360 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80804b2-503c-45b2-9db6-1bcc9538d080 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:08.007360 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80804b2-503c-45b2-9db6-1bcc9538d080 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:08.007564 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 274/536] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:56:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:08.007640 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80804b2-503c-45b2-9db6-1bcc9538d080 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:08.012452 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c0f8a-f685-45e1-a8ff-0f489b03f4db None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:08.015398 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80804b2-503c-45b2-9db6-1bcc9538d080 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:08.016904 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80804b2-503c-45b2-9db6-1bcc9538d080 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:08.028646 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-b80804b2-503c-45b2-9db6-1bcc9538d080 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:08.028646 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:08.028646 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:08.028646 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:08.028646 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:08.028646 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:08.028646 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:08.028646 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:08.028646 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:08.028646 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:08.028646 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:08.028646 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:08.028646 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:56:08.028646 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:56:08.028646 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:56:08.029481 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:56:08.029481 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:08.029481 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:08.029481 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:08.029481 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:08.029481 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:08.029481 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:08.029481 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:08.029481 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:08.029481 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:08.029481 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:08.029481 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:08.029481 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:08.029481 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:08.029481 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:08.029481 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:08.029481 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:08.030300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:56:08.030300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:56:08.030300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:56:08.030300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:56:08.030300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:56:08.030300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:56:08.030300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:08.030300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:08.030300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 263/537] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:56:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:08.035612 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80804b2-503c-45b2-9db6-1bcc9538d080 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:08.064875 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:08.082763 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:08.118932 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c0f8a-f685-45e1-a8ff-0f489b03f4db None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:08.186694 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2dc59-fccd-4d0c-a7d9-3ca3b219ff31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:08.187611 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2dc59-fccd-4d0c-a7d9-3ca3b219ff31 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:08.187898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2dc59-fccd-4d0c-a7d9-3ca3b219ff31 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:08.188333 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2dc59-fccd-4d0c-a7d9-3ca3b219ff31 None admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:08.189483 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b2dc59-fccd-4d0c-a7d9-3ca3b219ff31 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:08.189768 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b2dc59-fccd-4d0c-a7d9-3ca3b219ff31 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:08.191652 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b2dc59-fccd-4d0c-a7d9-3ca3b219ff31 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:08.206228 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80804b2-503c-45b2-9db6-1bcc9538d080 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:08.222781 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 275/538] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 13:56:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:08.224635 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2dc59-fccd-4d0c-a7d9-3ca3b219ff31 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:08.279038 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408cad7-48bb-44b5-b3c0-e025a20dcddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:08.279776 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408cad7-48bb-44b5-b3c0-e025a20dcddb None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:08.280066 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408cad7-48bb-44b5-b3c0-e025a20dcddb None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:08.280346 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408cad7-48bb-44b5-b3c0-e025a20dcddb None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:08.281136 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c408cad7-48bb-44b5-b3c0-e025a20dcddb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:08.281448 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c408cad7-48bb-44b5-b3c0-e025a20dcddb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:08.283081 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c408cad7-48bb-44b5-b3c0-e025a20dcddb None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:08.307417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 264/539] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 13:56:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:56:08.313183 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408cad7-48bb-44b5-b3c0-e025a20dcddb None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:08.332524 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2dc59-fccd-4d0c-a7d9-3ca3b219ff31 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:08.361412 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:08.367291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598f418-8a2c-4dc5-86fb-5363117aa094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:08.368658 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598f418-8a2c-4dc5-86fb-5363117aa094 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:08.369009 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598f418-8a2c-4dc5-86fb-5363117aa094 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:08.369362 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598f418-8a2c-4dc5-86fb-5363117aa094 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:08.370222 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598f418-8a2c-4dc5-86fb-5363117aa094 None admin] RBAC: Adding query filter params (role.id=d3d8169e71454005a8cf0d7544ad05dc, scope.project.id=72ecec9fad964b1fbeaefd0519597229, user.id=39a66f7362964389be039937a786bb0a) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:08.370601 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598f418-8a2c-4dc5-86fb-5363117aa094 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:08.372177 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598f418-8a2c-4dc5-86fb-5363117aa094 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:08.384655 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 276/540] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:56:08 2022] GET /identity/v3/role_assignments?user.id=39a66f7362964389be039937a786bb0a&scope.project.id=72ecec9fad964b1fbeaefd0519597229&role.id=d3d8169e71454005a8cf0d7544ad05dc => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:08.470374 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:08.511393 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408cad7-48bb-44b5-b3c0-e025a20dcddb None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:08.553392 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1440af3d-fb34-41f4-b5c6-43b6a8b7b56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:08.554026 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440af3d-fb34-41f4-b5c6-43b6a8b7b56f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:08.554312 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440af3d-fb34-41f4-b5c6-43b6a8b7b56f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:08.554589 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440af3d-fb34-41f4-b5c6-43b6a8b7b56f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:08.556025 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1440af3d-fb34-41f4-b5c6-43b6a8b7b56f None admin] RBAC: Adding query filter params (role.id=d3d8169e71454005a8cf0d7544ad05dc, scope.project.id=72ecec9fad964b1fbeaefd0519597229, user.id=fb9f698e09d246bcbea855613764fbe1) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:08.556339 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1440af3d-fb34-41f4-b5c6-43b6a8b7b56f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:08.557878 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1440af3d-fb34-41f4-b5c6-43b6a8b7b56f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:08.563905 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 265/541] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:56:08 2022] GET /identity/v3/role_assignments?user.id=fb9f698e09d246bcbea855613764fbe1&scope.project.id=72ecec9fad964b1fbeaefd0519597229&role.id=d3d8169e71454005a8cf0d7544ad05dc => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:56:08.649825 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a504f-6d2e-4b93-8ea9-0154303921f8 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:08.650496 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a504f-6d2e-4b93-8ea9-0154303921f8 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:08.650998 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a504f-6d2e-4b93-8ea9-0154303921f8 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:08.652834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 277/542] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:08.658363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec2b18-4624-4181-bbb6-372b44ad60f5 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:08.658647 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec2b18-4624-4181-bbb6-372b44ad60f5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:08.658933 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec2b18-4624-4181-bbb6-372b44ad60f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:08.680728 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-64ec2b18-4624-4181-bbb6-372b44ad60f5 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:08.681943 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-64ec2b18-4624-4181-bbb6-372b44ad60f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:08.784808 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 266/543] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:56:08.819832 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43a504f-6d2e-4b93-8ea9-0154303921f8 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:08.966754 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43a504f-6d2e-4b93-8ea9-0154303921f8 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:09.007985 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c7232-7acb-45e9-bd26-3df48500ae09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:09.009213 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c7232-7acb-45e9-bd26-3df48500ae09 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:09.009490 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c7232-7acb-45e9-bd26-3df48500ae09 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:09.009758 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c7232-7acb-45e9-bd26-3df48500ae09 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:09.020073 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c7232-7acb-45e9-bd26-3df48500ae09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:56:09.025451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-4f0c7232-7acb-45e9-bd26-3df48500ae09 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:56:09.025451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:09.025451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:09.025451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:09.025451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:09.025451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:09.025451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:09.025451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:09.025451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:09.025451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:09.025451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:09.025451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:09.025451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:56:09.025451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:56:09.025451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:56:09.025451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:56:09.026324 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:56:09.026324 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:56:09.026324 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:56:09.026324 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:56:09.026324 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:09.026324 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:09.026324 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:09.026324 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:09.026324 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:09.026324 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:09.026324 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:09.026324 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:09.026324 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:09.026324 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:09.026324 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:09.026324 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:09.027113 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:09.027113 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:09.027113 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:09.027113 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:09.027113 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:56:09.027113 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:56:09.027113 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:56:09.027113 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:56:09.027113 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:56:09.027113 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:56:09.027113 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:56:09.027113 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:09.027113 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 278/544] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 13:56:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:09.032556 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec2b18-4624-4181-bbb6-372b44ad60f5 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:09.090172 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d025e5-982b-43ec-b763-50ebbceaa5ae None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:09.090466 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d025e5-982b-43ec-b763-50ebbceaa5ae None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:09.090650 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d025e5-982b-43ec-b763-50ebbceaa5ae None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:09.092255 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 279/545] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:09.102939 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6490840-8059-40a0-a919-9ea0bcfc7512 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:09.103224 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6490840-8059-40a0-a919-9ea0bcfc7512 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:09.103500 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6490840-8059-40a0-a919-9ea0bcfc7512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:09.132727 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-f6490840-8059-40a0-a919-9ea0bcfc7512 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:09.134595 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-f6490840-8059-40a0-a919-9ea0bcfc7512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:09.143417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:09.180954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec2b18-4624-4181-bbb6-372b44ad60f5 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:09.209029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:09.229554 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9783bfc-4682-4fa8-b1eb-353e43b9c9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:09.230525 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9783bfc-4682-4fa8-b1eb-353e43b9c9ec None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:09.231063 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9783bfc-4682-4fa8-b1eb-353e43b9c9ec None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:09.231565 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9783bfc-4682-4fa8-b1eb-353e43b9c9ec None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:09.232684 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9783bfc-4682-4fa8-b1eb-353e43b9c9ec None admin] RBAC: Adding query filter params (name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:09.233231 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9783bfc-4682-4fa8-b1eb-353e43b9c9ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:09.235025 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9783bfc-4682-4fa8-b1eb-353e43b9c9ec None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:09.245830 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 267/546] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 13:56:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:09.251569 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9783bfc-4682-4fa8-b1eb-353e43b9c9ec None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:09.252019 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 280/547] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:56:09.306501 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6490840-8059-40a0-a919-9ea0bcfc7512 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:09.415375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9783bfc-4682-4fa8-b1eb-353e43b9c9ec None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:09.445627 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:09.461549 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35678c2-5bfa-44a6-a753-4970673dff7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:09.463441 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35678c2-5bfa-44a6-a753-4970673dff7b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:09.464512 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35678c2-5bfa-44a6-a753-4970673dff7b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:09.465017 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35678c2-5bfa-44a6-a753-4970673dff7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:09.489389 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35678c2-5bfa-44a6-a753-4970673dff7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:09.492390 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35678c2-5bfa-44a6-a753-4970673dff7b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:09.519409 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-d35678c2-5bfa-44a6-a753-4970673dff7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:09.519409 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:09.519409 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:09.519409 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:09.519409 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:09.519409 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:09.519409 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:09.519409 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:09.519409 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:09.519409 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:09.519409 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:09.519409 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:09.519409 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:09.519409 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:09.519409 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:09.520282 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:09.520282 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:09.520282 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:09.520282 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:09.520282 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:09.520282 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:09.520282 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:09.520282 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:09.520282 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:09.520282 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:09.520282 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:09.520282 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:09.520282 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:09.520282 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:09.520282 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:09.520282 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:09.520282 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:09.521118 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:09.521118 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:09.521118 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:09.521118 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:09.529614 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6490840-8059-40a0-a919-9ea0bcfc7512 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:09.531149 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 268/548] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:56:09.536627 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35678c2-5bfa-44a6-a753-4970673dff7b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:09.584670 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834a74c-fbcc-4a8d-89ec-7ba8666d8894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:09.585674 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834a74c-fbcc-4a8d-89ec-7ba8666d8894 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:09.586176 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834a74c-fbcc-4a8d-89ec-7ba8666d8894 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:09.586698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834a74c-fbcc-4a8d-89ec-7ba8666d8894 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:09.597657 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4834a74c-fbcc-4a8d-89ec-7ba8666d8894 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:56:09.603593 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-4834a74c-fbcc-4a8d-89ec-7ba8666d8894 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:56:09.603593 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:09.603593 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:09.603593 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:09.603593 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:09.603593 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:09.603593 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:09.603593 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:09.603593 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:09.603593 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:09.603593 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:09.603593 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:09.603593 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:56:09.603593 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:56:09.603593 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:56:09.603593 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:56:09.604444 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:56:09.604444 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:56:09.604444 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:56:09.604444 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:56:09.604444 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:09.604444 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:09.604444 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:09.604444 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:09.604444 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:09.604444 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:09.604444 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:09.604444 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:09.604444 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:09.604444 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:09.604444 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:09.604444 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:09.605786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:09.605786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:09.605786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:09.605786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:09.605786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:56:09.605786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:56:09.605786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:56:09.605786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:56:09.605786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:56:09.605786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:56:09.605786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:56:09.605786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:09.612711 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 281/549] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 13:56:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:09.614531 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834a74c-fbcc-4a8d-89ec-7ba8666d8894 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:09.663384 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:09.718622 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35678c2-5bfa-44a6-a753-4970673dff7b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:09.729718 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834a74c-fbcc-4a8d-89ec-7ba8666d8894 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:09.777047 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019af8b-63ce-4cf6-8472-3277f0fa8db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:09.778401 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019af8b-63ce-4cf6-8472-3277f0fa8db6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:09.778911 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019af8b-63ce-4cf6-8472-3277f0fa8db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:09.779388 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019af8b-63ce-4cf6-8472-3277f0fa8db6 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:09.780310 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a019af8b-63ce-4cf6-8472-3277f0fa8db6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:09.780869 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a019af8b-63ce-4cf6-8472-3277f0fa8db6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:09.782710 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a019af8b-63ce-4cf6-8472-3277f0fa8db6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:09.797315 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 269/550] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:56:09.801880 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e472c98-c13c-47fa-9c2c-eea179ec0744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:09.802536 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e472c98-c13c-47fa-9c2c-eea179ec0744 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:09.802820 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e472c98-c13c-47fa-9c2c-eea179ec0744 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:09.803099 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e472c98-c13c-47fa-9c2c-eea179ec0744 None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:09.803838 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e472c98-c13c-47fa-9c2c-eea179ec0744 None admin] RBAC: Adding query filter params (name=service) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:09.804146 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e472c98-c13c-47fa-9c2c-eea179ec0744 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:09.806076 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019af8b-63ce-4cf6-8472-3277f0fa8db6 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:09.806192 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e472c98-c13c-47fa-9c2c-eea179ec0744 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:09.827319 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 282/551] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 13:56:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:56:09.832891 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e472c98-c13c-47fa-9c2c-eea179ec0744 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:09.884826 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:09.960966 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e472c98-c13c-47fa-9c2c-eea179ec0744 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:09.964224 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:09.972440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019af8b-63ce-4cf6-8472-3277f0fa8db6 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:09.997203 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ddc472-27a7-4ace-8d4a-f08f1aad9ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:09.997851 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddc472-27a7-4ace-8d4a-f08f1aad9ffe None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:09.998137 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddc472-27a7-4ace-8d4a-f08f1aad9ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:09.998407 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddc472-27a7-4ace-8d4a-f08f1aad9ffe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:10.009594 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ddc472-27a7-4ace-8d4a-f08f1aad9ffe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:10.010340 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efff198-561c-4404-b46b-d7edb3862f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:10.011061 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ddc472-27a7-4ace-8d4a-f08f1aad9ffe None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:10.011176 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efff198-561c-4404-b46b-d7edb3862f7a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:10.011715 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efff198-561c-4404-b46b-d7edb3862f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:10.012108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efff198-561c-4404-b46b-d7edb3862f7a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:10.020627 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efff198-561c-4404-b46b-d7edb3862f7a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:10.022327 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efff198-561c-4404-b46b-d7edb3862f7a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:10.026411 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-73ddc472-27a7-4ace-8d4a-f08f1aad9ffe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:10.026411 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:10.026411 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:10.026411 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:10.026411 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:10.026411 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:10.026411 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:10.026411 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:10.026411 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:10.026411 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:10.026411 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:10.026411 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:10.026411 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:10.026411 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:10.026411 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:10.027328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:10.027328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:10.027328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:10.027328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:10.027328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:10.027328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:10.027328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:10.027328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:10.027328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:10.027328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:10.027328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:10.027328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:10.027328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:10.027328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:10.027328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:10.027328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:10.027328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:10.028325 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:10.028325 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:10.028325 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:10.028325 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:10.028325 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 283/552] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:56:10.030446 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a79665b-af16-44cd-98ed-58c5c94f0df9 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:10.030446 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a79665b-af16-44cd-98ed-58c5c94f0df9 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:10.030446 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a79665b-af16-44cd-98ed-58c5c94f0df9 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:10.030900 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 284/553] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:10.035782 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a79665b-af16-44cd-98ed-58c5c94f0df9 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:10.037281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-9efff198-561c-4404-b46b-d7edb3862f7a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 13:56:10.037281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:10.037281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:10.037281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:10.037281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:10.037281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:10.037281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:10.037281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:10.037281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:10.037281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:10.037281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:10.037281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:10.037281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:56:10.037281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:56:10.037281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:56:10.037281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:56:10.039046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:10.039046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:10.039046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:56:10.039046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:10.039046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:56:10.039046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:10.039046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:10.039046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:10.039046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:10.039046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:10.039046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:10.039046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:10.039046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:10.039046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:10.039046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:10.039046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:10.039046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:10.039046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:10.041436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:10.041436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:10.041436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:56:10.041436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:56:10.041436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:56:10.041436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:56:10.041436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:56:10.041436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:56:10.041436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 13:56:10.041436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:10.041436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 270/554] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 13:56:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:10.048503 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc3345-400f-4b60-8df0-eacad3413285 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:10.048503 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc3345-400f-4b60-8df0-eacad3413285 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:10.048503 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc3345-400f-4b60-8df0-eacad3413285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:10.062972 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4fbc3345-400f-4b60-8df0-eacad3413285 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:10.064273 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4fbc3345-400f-4b60-8df0-eacad3413285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:10.150788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a79665b-af16-44cd-98ed-58c5c94f0df9 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:10.160722 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 271/555] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:56:10.161612 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc3345-400f-4b60-8df0-eacad3413285 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:10.191750 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a300e7fd-2f0a-478d-937b-deb8cfb4d29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:10.192599 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300e7fd-2f0a-478d-937b-deb8cfb4d29e None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:10.193012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300e7fd-2f0a-478d-937b-deb8cfb4d29e None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:10.193292 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300e7fd-2f0a-478d-937b-deb8cfb4d29e None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:10.194250 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a300e7fd-2f0a-478d-937b-deb8cfb4d29e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:10.194559 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a300e7fd-2f0a-478d-937b-deb8cfb4d29e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:10.196639 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a300e7fd-2f0a-478d-937b-deb8cfb4d29e None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:10.207414 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:10.215519 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 285/556] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:10.216457 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a300e7fd-2f0a-478d-937b-deb8cfb4d29e None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:10.240988 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:10.293732 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc3345-400f-4b60-8df0-eacad3413285 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:10.336064 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a300e7fd-2f0a-478d-937b-deb8cfb4d29e None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:10.341850 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4363cd31-fe0b-4a60-bd5a-3bb829862524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:10.342485 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363cd31-fe0b-4a60-bd5a-3bb829862524 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:10.342771 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363cd31-fe0b-4a60-bd5a-3bb829862524 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:10.343049 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363cd31-fe0b-4a60-bd5a-3bb829862524 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:10.344384 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4363cd31-fe0b-4a60-bd5a-3bb829862524 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:10.344728 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4363cd31-fe0b-4a60-bd5a-3bb829862524 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:10.346709 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4363cd31-fe0b-4a60-bd5a-3bb829862524 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:10.371878 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 272/557] 10.222.0.48 () {56 vars in 1135 bytes} [Mon Aug 1 13:56:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:10.372852 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4363cd31-fe0b-4a60-bd5a-3bb829862524 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:10.378311 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc9e82c-7a17-48f6-beae-970774babe9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:10.378971 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9e82c-7a17-48f6-beae-970774babe9f None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:10.379254 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9e82c-7a17-48f6-beae-970774babe9f None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:10.379533 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9e82c-7a17-48f6-beae-970774babe9f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:10.393659 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc9e82c-7a17-48f6-beae-970774babe9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:56:10.402037 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-3fc9e82c-7a17-48f6-beae-970774babe9f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:56:10.402037 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:10.402037 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:10.402037 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:10.402037 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:10.402037 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:10.402037 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:10.402037 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:10.402037 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:10.402037 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:10.402037 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:10.402037 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:10.402037 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:56:10.402037 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:56:10.402037 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:56:10.402037 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:56:10.403516 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:56:10.403516 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:56:10.403516 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:56:10.403516 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:56:10.403516 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:10.403516 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:10.403516 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:10.403516 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:10.403516 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:10.403516 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:10.403516 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:10.403516 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:10.403516 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:10.403516 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:10.403516 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:10.403516 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:10.404376 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:10.404376 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:10.404376 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:10.404376 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:10.404376 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:56:10.404376 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:56:10.404376 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:56:10.404376 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:56:10.404376 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:56:10.404376 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:56:10.404376 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:56:10.404376 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:10.408517 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 286/558] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 13:56:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:10.408517 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc9e82c-7a17-48f6-beae-970774babe9f None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:10.497696 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:10.499367 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:10.515162 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4363cd31-fe0b-4a60-bd5a-3bb829862524 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:10.531850 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc9e82c-7a17-48f6-beae-970774babe9f None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:10.554814 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6440ba8b-d44c-4b9f-b3b1-0d6b7671eac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:10.556075 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6440ba8b-d44c-4b9f-b3b1-0d6b7671eac6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:10.556361 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6440ba8b-d44c-4b9f-b3b1-0d6b7671eac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:10.556670 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6440ba8b-d44c-4b9f-b3b1-0d6b7671eac6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:10.564716 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6440ba8b-d44c-4b9f-b3b1-0d6b7671eac6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:10.566353 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6440ba8b-d44c-4b9f-b3b1-0d6b7671eac6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:10.575211 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796997d4-3b90-4d5b-a664-ce1b2737b8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:10.575846 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796997d4-3b90-4d5b-a664-ce1b2737b8aa None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:10.576129 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796997d4-3b90-4d5b-a664-ce1b2737b8aa None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:10.576406 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796997d4-3b90-4d5b-a664-ce1b2737b8aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:10.579417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-6440ba8b-d44c-4b9f-b3b1-0d6b7671eac6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 13:56:10.579417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:10.579417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:10.579417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:10.579417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:10.579417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:10.579417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:10.579417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:10.579417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:10.579417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:10.579417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:10.579417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:10.579417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:56:10.579417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:56:10.579417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:56:10.579417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:56:10.581580 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:10.581580 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:10.581580 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:56:10.581580 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:10.581580 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:56:10.581580 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:10.581580 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:10.581580 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:10.581580 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:10.581580 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:10.581580 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:10.581580 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:10.581580 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:10.581580 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:10.581580 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:10.581580 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:10.581580 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:10.581580 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:10.583759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:10.583759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:10.583759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:56:10.583759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:56:10.583759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:56:10.583759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:56:10.583759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:56:10.583759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:56:10.583759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 13:56:10.583759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:10.583759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 273/559] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Aug 1 13:56:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:10.583759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6440ba8b-d44c-4b9f-b3b1-0d6b7671eac6 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:10.663788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796997d4-3b90-4d5b-a664-ce1b2737b8aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:10.665337 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796997d4-3b90-4d5b-a664-ce1b2737b8aa None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:10.698303 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6440ba8b-d44c-4b9f-b3b1-0d6b7671eac6 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:10.735304 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2645947-c6bc-4512-8bb0-45200525c9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:10.735928 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2645947-c6bc-4512-8bb0-45200525c9cd None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:10.736210 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2645947-c6bc-4512-8bb0-45200525c9cd None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:10.736507 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2645947-c6bc-4512-8bb0-45200525c9cd None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:10.737232 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2645947-c6bc-4512-8bb0-45200525c9cd None admin] RBAC: Adding query filter params (name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:10.737539 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2645947-c6bc-4512-8bb0-45200525c9cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:10.738899 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2645947-c6bc-4512-8bb0-45200525c9cd None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:10.753732 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 274/560] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 13:56:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:10.753732 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2645947-c6bc-4512-8bb0-45200525c9cd None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:10.769255 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:10.771929 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:10.806853 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 287/561] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:56:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:56:10.808108 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b062d7-3653-42ac-a01e-2ab0d0e61ec8 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:10.808413 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b062d7-3653-42ac-a01e-2ab0d0e61ec8 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:10.808739 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b062d7-3653-42ac-a01e-2ab0d0e61ec8 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:10.810192 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 288/562] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:10.810762 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b062d7-3653-42ac-a01e-2ab0d0e61ec8 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:10.889398 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2645947-c6bc-4512-8bb0-45200525c9cd None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:10.927161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec1253f-725a-462f-a29c-fc4fe31dd854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:10.927819 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1253f-725a-462f-a29c-fc4fe31dd854 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:10.928077 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1253f-725a-462f-a29c-fc4fe31dd854 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:10.928363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1253f-725a-462f-a29c-fc4fe31dd854 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:10.929128 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b062d7-3653-42ac-a01e-2ab0d0e61ec8 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:10.929555 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec1253f-725a-462f-a29c-fc4fe31dd854 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:10.929555 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec1253f-725a-462f-a29c-fc4fe31dd854 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:10.930966 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec1253f-725a-462f-a29c-fc4fe31dd854 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:10.949847 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 275/563] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Aug 1 13:56:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:10.951005 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec1253f-725a-462f-a29c-fc4fe31dd854 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:10.973245 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee12c45-dc1e-4737-9128-801ecbbc9173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:10.973936 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee12c45-dc1e-4737-9128-801ecbbc9173 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:10.974232 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee12c45-dc1e-4737-9128-801ecbbc9173 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:10.974511 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee12c45-dc1e-4737-9128-801ecbbc9173 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:10.985837 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee12c45-dc1e-4737-9128-801ecbbc9173 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:10.987475 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee12c45-dc1e-4737-9128-801ecbbc9173 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:11.003184 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-eee12c45-dc1e-4737-9128-801ecbbc9173 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:11.003184 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:11.003184 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:11.003184 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:11.003184 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:11.003184 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:11.003184 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:11.003184 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:11.003184 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:11.003184 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:11.003184 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:11.003184 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:11.003184 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:11.003184 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:11.003184 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:11.004434 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:11.004434 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:11.004434 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:11.004434 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:11.004434 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:11.004434 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:11.004434 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:11.004434 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:11.004434 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:11.004434 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:11.004434 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:11.004434 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:11.004434 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:11.004434 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:11.004434 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:11.004434 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:11.004434 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:11.005583 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:11.005583 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:11.005583 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:11.005583 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:11.009611 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 289/564] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:11.010718 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468cbce-5d6b-487f-8666-b1739d82beb6 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:11.011007 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468cbce-5d6b-487f-8666-b1739d82beb6 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:11.011283 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468cbce-5d6b-487f-8666-b1739d82beb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:11.020073 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:11.031873 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-5468cbce-5d6b-487f-8666-b1739d82beb6 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:11.032762 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-5468cbce-5d6b-487f-8666-b1739d82beb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:11.054998 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:11.055857 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec1253f-725a-462f-a29c-fc4fe31dd854 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:11.090677 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4d347-89ab-455d-90ba-20a0e357efa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:11.091308 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4d347-89ab-455d-90ba-20a0e357efa6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:11.091590 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4d347-89ab-455d-90ba-20a0e357efa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:11.091864 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4d347-89ab-455d-90ba-20a0e357efa6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:11.101764 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4d347-89ab-455d-90ba-20a0e357efa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:11.103210 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4d347-89ab-455d-90ba-20a0e357efa6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:11.118853 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-2bb4d347-89ab-455d-90ba-20a0e357efa6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:11.118853 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:11.118853 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:11.118853 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:11.118853 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:11.118853 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:11.118853 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:11.118853 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:11.118853 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:11.118853 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:11.118853 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:11.118853 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:11.118853 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:11.118853 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:11.118853 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:11.119954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:11.119954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:11.119954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:11.119954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:11.119954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:11.119954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:11.119954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:11.119954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:11.119954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:11.119954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:11.119954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:11.119954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:11.119954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:11.119954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:11.119954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:11.119954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:11.119954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:11.120868 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:11.120868 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:11.120868 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:11.120868 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:11.120868 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 276/565] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:11.120868 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4d347-89ab-455d-90ba-20a0e357efa6 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:11.124216 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 290/566] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:56:11.125818 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b28d57-d545-4382-a16a-a870d475ef7d None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:11.126108 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b28d57-d545-4382-a16a-a870d475ef7d None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:11.126389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b28d57-d545-4382-a16a-a870d475ef7d None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:11.131064 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 291/567] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:11.131064 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b28d57-d545-4382-a16a-a870d475ef7d None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:11.232783 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4d347-89ab-455d-90ba-20a0e357efa6 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:11.246021 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b28d57-d545-4382-a16a-a870d475ef7d None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:11.252300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:11.267786 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e68fc-c441-4df6-9827-3ec696aea9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:11.268348 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e68fc-c441-4df6-9827-3ec696aea9d1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:11.268659 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e68fc-c441-4df6-9827-3ec696aea9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:11.268937 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e68fc-c441-4df6-9827-3ec696aea9d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:11.279905 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889e68fc-c441-4df6-9827-3ec696aea9d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:11.281377 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889e68fc-c441-4df6-9827-3ec696aea9d1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:11.281859 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f485e9-e39a-40c2-8d95-8c7e65263968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:11.282479 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f485e9-e39a-40c2-8d95-8c7e65263968 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:11.282766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f485e9-e39a-40c2-8d95-8c7e65263968 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:11.283046 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f485e9-e39a-40c2-8d95-8c7e65263968 None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:11.283742 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f485e9-e39a-40c2-8d95-8c7e65263968 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:11.284049 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f485e9-e39a-40c2-8d95-8c7e65263968 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:11.285436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f485e9-e39a-40c2-8d95-8c7e65263968 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:11.296128 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-889e68fc-c441-4df6-9827-3ec696aea9d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:11.296128 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:11.296128 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:11.296128 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:11.296128 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:11.296128 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:11.296128 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:11.296128 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:11.296128 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:11.296128 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:11.296128 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:11.296128 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:11.296128 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:11.296128 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:11.296128 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:11.297043 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:11.297043 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:11.297043 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:11.297043 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:11.297043 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:11.297043 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:11.297043 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:11.297043 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:11.297043 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:11.297043 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:11.297043 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:11.297043 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:11.297043 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:11.297043 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:11.297043 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:11.297043 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:11.297043 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:11.297912 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:11.297912 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:11.297912 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:11.297912 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:11.298135 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 292/568] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:11.298135 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f485e9-e39a-40c2-8d95-8c7e65263968 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:11.300613 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 277/569] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:11.302027 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59942ced-14f2-459d-b946-54fc31f59c9b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:11.302488 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59942ced-14f2-459d-b946-54fc31f59c9b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:11.302764 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59942ced-14f2-459d-b946-54fc31f59c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:11.320392 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:11.321562 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-59942ced-14f2-459d-b946-54fc31f59c9b None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:11.322735 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-59942ced-14f2-459d-b946-54fc31f59c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:11.387996 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 278/570] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:56:11.389051 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59942ced-14f2-459d-b946-54fc31f59c9b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:11.400222 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f485e9-e39a-40c2-8d95-8c7e65263968 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:11.435840 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914cf99-356d-4c4e-a43c-a6e7dd14556c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:11.436505 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914cf99-356d-4c4e-a43c-a6e7dd14556c None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:11.436802 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914cf99-356d-4c4e-a43c-a6e7dd14556c None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:11.437081 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914cf99-356d-4c4e-a43c-a6e7dd14556c None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:11.437784 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e914cf99-356d-4c4e-a43c-a6e7dd14556c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:11.438096 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e914cf99-356d-4c4e-a43c-a6e7dd14556c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:11.439497 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e914cf99-356d-4c4e-a43c-a6e7dd14556c None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:11.451336 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 293/571] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:11.452170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914cf99-356d-4c4e-a43c-a6e7dd14556c None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:11.507032 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59942ced-14f2-459d-b946-54fc31f59c9b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:11.509402 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:11.537227 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:11.556778 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35951e-47d2-4045-8725-5eabfb60fbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:11.557729 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35951e-47d2-4045-8725-5eabfb60fbdd None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:11.558158 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35951e-47d2-4045-8725-5eabfb60fbdd None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:11.558572 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35951e-47d2-4045-8725-5eabfb60fbdd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:11.560741 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914cf99-356d-4c4e-a43c-a6e7dd14556c None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:11.573017 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab35951e-47d2-4045-8725-5eabfb60fbdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:56:11.580498 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-ab35951e-47d2-4045-8725-5eabfb60fbdd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:56:11.580498 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:11.580498 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:11.580498 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:11.580498 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:11.580498 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:11.580498 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:11.580498 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:11.580498 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:11.580498 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:11.580498 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:11.580498 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:11.580498 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:56:11.580498 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:56:11.580498 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:56:11.580498 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:56:11.581609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:56:11.581609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:56:11.581609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:56:11.581609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:56:11.581609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:11.581609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:11.581609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:11.581609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:11.581609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:11.581609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:11.581609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:11.581609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:11.581609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:11.581609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:11.581609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:11.581609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:11.582634 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:11.582634 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:11.582634 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:11.582634 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:11.582634 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:56:11.582634 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:56:11.582634 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:56:11.582634 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:56:11.582634 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:56:11.582634 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:56:11.582634 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:56:11.582634 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:11.582634 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 279/572] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 13:56:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:11.583462 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35951e-47d2-4045-8725-5eabfb60fbdd None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:11.599348 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53421347-d588-4ff1-a72d-03c0d6112bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:11.599991 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53421347-d588-4ff1-a72d-03c0d6112bdf None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:11.600284 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53421347-d588-4ff1-a72d-03c0d6112bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:11.600592 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53421347-d588-4ff1-a72d-03c0d6112bdf None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:11.607866 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53421347-d588-4ff1-a72d-03c0d6112bdf None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:11.609344 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53421347-d588-4ff1-a72d-03c0d6112bdf None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:11.620664 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-53421347-d588-4ff1-a72d-03c0d6112bdf None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 13:56:11.620664 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:11.620664 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:11.620664 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:11.620664 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:11.620664 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:11.620664 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:11.620664 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:11.620664 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:11.620664 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:11.620664 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:11.620664 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:11.620664 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:56:11.620664 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:56:11.620664 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:56:11.620664 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:56:11.621756 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:11.621756 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:11.621756 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:56:11.621756 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:11.621756 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:56:11.621756 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:11.621756 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:11.621756 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:11.621756 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:11.621756 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:11.621756 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:11.621756 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:11.621756 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:11.621756 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:11.621756 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:11.621756 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:11.621756 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:11.621756 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:11.622929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:11.622929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:11.622929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:56:11.622929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:56:11.622929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:56:11.622929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:56:11.622929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:56:11.622929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:56:11.622929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 13:56:11.622929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:11.622929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 294/573] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:56:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:11.622929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53421347-d588-4ff1-a72d-03c0d6112bdf None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:11.688987 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35951e-47d2-4045-8725-5eabfb60fbdd None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:11.723350 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7730d5-16ae-421a-bab1-5892abaaf03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:11.723893 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53421347-d588-4ff1-a72d-03c0d6112bdf None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:11.724035 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7730d5-16ae-421a-bab1-5892abaaf03e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:11.724232 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7730d5-16ae-421a-bab1-5892abaaf03e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:11.724533 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7730d5-16ae-421a-bab1-5892abaaf03e None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:11.725236 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7730d5-16ae-421a-bab1-5892abaaf03e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:11.725542 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7730d5-16ae-421a-bab1-5892abaaf03e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:11.726946 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7730d5-16ae-421a-bab1-5892abaaf03e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:11.738019 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 280/574] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:11.738801 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7730d5-16ae-421a-bab1-5892abaaf03e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:11.756840 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7c5d7-47cc-422c-aeb9-cf33ca5389e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:11.757485 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7c5d7-47cc-422c-aeb9-cf33ca5389e9 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:11.757770 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7c5d7-47cc-422c-aeb9-cf33ca5389e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:11.758576 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7c5d7-47cc-422c-aeb9-cf33ca5389e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:11.762714 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:11.766420 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:11.773249 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec7c5d7-47cc-422c-aeb9-cf33ca5389e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:56:11.778177 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-cec7c5d7-47cc-422c-aeb9-cf33ca5389e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:56:11.778177 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:11.778177 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:11.778177 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:11.778177 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:11.778177 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:11.778177 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:11.778177 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:11.778177 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:11.778177 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:11.778177 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:11.778177 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:11.778177 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:56:11.778177 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:56:11.778177 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:56:11.778177 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:56:11.779290 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:56:11.779290 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:56:11.779290 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:56:11.779290 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:56:11.779290 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:11.779290 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:11.779290 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:11.779290 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:11.779290 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:11.779290 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:11.779290 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:11.779290 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:11.779290 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:11.779290 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:11.779290 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:11.779290 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:11.780136 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:11.780136 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:11.780136 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:11.780136 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:11.780136 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:56:11.780136 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:56:11.780136 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:56:11.780136 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:56:11.780136 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:56:11.780136 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:56:11.780136 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:56:11.780136 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:11.780136 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 295/575] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 13:56:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:11.780136 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7c5d7-47cc-422c-aeb9-cf33ca5389e9 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:11.842481 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7730d5-16ae-421a-bab1-5892abaaf03e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:11.882521 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9efcd3-f098-43db-b9fd-069095dfc307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:11.883302 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9efcd3-f098-43db-b9fd-069095dfc307 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:11.883414 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7c5d7-47cc-422c-aeb9-cf33ca5389e9 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:11.883593 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9efcd3-f098-43db-b9fd-069095dfc307 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:11.883916 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9efcd3-f098-43db-b9fd-069095dfc307 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:11.891872 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9efcd3-f098-43db-b9fd-069095dfc307 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:11.893371 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9efcd3-f098-43db-b9fd-069095dfc307 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:11.906439 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-6f9efcd3-f098-43db-b9fd-069095dfc307 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:11.906439 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:11.906439 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:11.906439 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:11.906439 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:11.906439 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:11.906439 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:11.906439 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:11.906439 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:11.906439 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:11.906439 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:11.906439 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:11.906439 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:56:11.906439 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:56:11.906439 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:56:11.907349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:56:11.907349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:11.907349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:11.907349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:11.907349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:11.907349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:11.907349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:11.907349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:11.907349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:11.907349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:11.907349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:11.907349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:11.907349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:11.907349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:11.907349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:11.907349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:11.907349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:11.908198 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:56:11.908198 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:56:11.908198 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:56:11.908198 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:56:11.908198 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:56:11.908198 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:56:11.908198 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:11.908198 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:11.908198 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 281/576] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:56:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:11.909159 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9efcd3-f098-43db-b9fd-069095dfc307 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:11.923008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e879f-8ab2-4672-b49d-6797286edd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:11.923640 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e879f-8ab2-4672-b49d-6797286edd16 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:11.923905 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e879f-8ab2-4672-b49d-6797286edd16 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:11.924176 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e879f-8ab2-4672-b49d-6797286edd16 None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:11.924923 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e879f-8ab2-4672-b49d-6797286edd16 None admin] RBAC: Adding query filter params (name=service) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:11.925222 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e879f-8ab2-4672-b49d-6797286edd16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:11.926599 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e879f-8ab2-4672-b49d-6797286edd16 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:11.935304 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 296/577] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 13:56:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:11.936075 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e879f-8ab2-4672-b49d-6797286edd16 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:11.989866 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:11.998476 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:12.008827 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9efcd3-f098-43db-b9fd-069095dfc307 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:12.039627 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e879f-8ab2-4672-b49d-6797286edd16 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:12.041953 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07e6643-c18b-4bd9-9f5c-66c07e136322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:12.042572 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e6643-c18b-4bd9-9f5c-66c07e136322 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:12.042856 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e6643-c18b-4bd9-9f5c-66c07e136322 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:12.043134 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e6643-c18b-4bd9-9f5c-66c07e136322 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:12.043928 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07e6643-c18b-4bd9-9f5c-66c07e136322 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:12.044239 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07e6643-c18b-4bd9-9f5c-66c07e136322 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:12.045699 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07e6643-c18b-4bd9-9f5c-66c07e136322 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:12.062910 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 282/578] 10.222.0.48 () {56 vars in 1137 bytes} [Mon Aug 1 13:56:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:12.063704 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07e6643-c18b-4bd9-9f5c-66c07e136322 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:12.074065 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f67854-24cb-4f3c-a2d7-9c11b5f95b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:12.074642 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f67854-24cb-4f3c-a2d7-9c11b5f95b5e None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:12.074921 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f67854-24cb-4f3c-a2d7-9c11b5f95b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:12.075197 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f67854-24cb-4f3c-a2d7-9c11b5f95b5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:12.081814 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f67854-24cb-4f3c-a2d7-9c11b5f95b5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:12.083240 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f67854-24cb-4f3c-a2d7-9c11b5f95b5e None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:12.094581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-35f67854-24cb-4f3c-a2d7-9c11b5f95b5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:12.094581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:12.094581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:12.094581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:12.094581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:12.094581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:12.094581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:12.094581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:12.094581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:12.094581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:12.094581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:12.094581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:12.094581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:56:12.094581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:56:12.094581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:56:12.095423 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:56:12.095423 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:12.095423 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:12.095423 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:12.095423 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:12.095423 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:12.095423 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:12.095423 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:12.095423 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:12.095423 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:12.095423 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:12.095423 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:12.095423 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:12.095423 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:12.095423 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:12.095423 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:12.095423 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:12.096276 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:56:12.096276 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:56:12.096276 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:56:12.096276 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:56:12.096276 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:56:12.096276 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:56:12.096276 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:12.096276 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:12.096276 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 297/579] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:56:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:12.096812 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f67854-24cb-4f3c-a2d7-9c11b5f95b5e None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:12.164076 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07e6643-c18b-4bd9-9f5c-66c07e136322 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:12.197289 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b4266f-e8d7-44e2-a7eb-6c2f7dca89e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:12.197902 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4266f-e8d7-44e2-a7eb-6c2f7dca89e4 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:12.198048 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f67854-24cb-4f3c-a2d7-9c11b5f95b5e None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:12.198172 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4266f-e8d7-44e2-a7eb-6c2f7dca89e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:12.198443 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4266f-e8d7-44e2-a7eb-6c2f7dca89e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:12.199153 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b4266f-e8d7-44e2-a7eb-6c2f7dca89e4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:12.199456 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b4266f-e8d7-44e2-a7eb-6c2f7dca89e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:12.200843 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b4266f-e8d7-44e2-a7eb-6c2f7dca89e4 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:12.208519 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 283/580] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 13:56:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:12.209296 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b4266f-e8d7-44e2-a7eb-6c2f7dca89e4 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:12.212392 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:12.219041 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:12.234617 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a9195-5fcb-4903-9b05-37a6f0f37051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:12.235231 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a9195-5fcb-4903-9b05-37a6f0f37051 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:12.235534 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a9195-5fcb-4903-9b05-37a6f0f37051 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:12.235894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a9195-5fcb-4903-9b05-37a6f0f37051 None admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:12.236718 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a9195-5fcb-4903-9b05-37a6f0f37051 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:12.237027 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a9195-5fcb-4903-9b05-37a6f0f37051 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:12.238540 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a9195-5fcb-4903-9b05-37a6f0f37051 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:12.250182 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 298/581] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 13:56:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:12.251038 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a9195-5fcb-4903-9b05-37a6f0f37051 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:12.312658 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b4266f-e8d7-44e2-a7eb-6c2f7dca89e4 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:12.346049 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb868cc-2d9a-4810-9989-a7ebeccf1d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:12.346623 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb868cc-2d9a-4810-9989-a7ebeccf1d64 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:12.346904 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb868cc-2d9a-4810-9989-a7ebeccf1d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:12.347179 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb868cc-2d9a-4810-9989-a7ebeccf1d64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:12.350967 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a9195-5fcb-4903-9b05-37a6f0f37051 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:12.358129 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb868cc-2d9a-4810-9989-a7ebeccf1d64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:12.359558 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb868cc-2d9a-4810-9989-a7ebeccf1d64 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:12.373771 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-4eb868cc-2d9a-4810-9989-a7ebeccf1d64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:12.373771 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:12.373771 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:12.373771 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:12.373771 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:12.373771 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:12.373771 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:12.373771 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:12.373771 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:12.373771 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:12.373771 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:12.373771 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:12.373771 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:12.373771 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:12.373771 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:12.374731 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:12.374731 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:12.374731 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:12.374731 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:12.374731 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:12.374731 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:12.374731 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:12.374731 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:12.374731 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:12.374731 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:12.374731 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:12.374731 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:12.374731 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:12.374731 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:12.374731 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:12.374731 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:12.374731 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:12.375608 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:12.375608 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:12.375608 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:12.375608 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:12.375608 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 284/582] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:12.375877 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb868cc-2d9a-4810-9989-a7ebeccf1d64 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:12.390299 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bc9030-444d-42d1-bc14-0dfd3c436eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:12.390941 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc9030-444d-42d1-bc14-0dfd3c436eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:12.391231 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc9030-444d-42d1-bc14-0dfd3c436eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:12.391512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc9030-444d-42d1-bc14-0dfd3c436eb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:12.402591 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bc9030-444d-42d1-bc14-0dfd3c436eb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:12.404049 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bc9030-444d-42d1-bc14-0dfd3c436eb1 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:12.418424 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-c3bc9030-444d-42d1-bc14-0dfd3c436eb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:12.418424 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:12.418424 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:12.418424 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:12.418424 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:12.418424 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:12.418424 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:12.418424 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:12.418424 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:12.418424 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:12.418424 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:12.418424 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:12.418424 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:12.418424 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:12.418424 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:12.419266 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:12.419266 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:12.419266 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:12.419266 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:12.419266 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:12.419266 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:12.419266 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:12.419266 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:12.419266 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:12.419266 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:12.419266 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:12.419266 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:12.419266 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:12.419266 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:12.419266 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:12.419266 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:12.419266 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:12.420303 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:12.420303 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:12.420303 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:12.420303 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:12.420303 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 299/583] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:12.420303 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bc9030-444d-42d1-bc14-0dfd3c436eb1 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:12.432980 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:12.453013 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:12.478919 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb868cc-2d9a-4810-9989-a7ebeccf1d64 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:12.513839 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2dd0b8-c8a7-41ec-943e-de3645b440ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:12.514477 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2dd0b8-c8a7-41ec-943e-de3645b440ae None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:12.514786 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2dd0b8-c8a7-41ec-943e-de3645b440ae None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:12.515070 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2dd0b8-c8a7-41ec-943e-de3645b440ae None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:12.515824 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2dd0b8-c8a7-41ec-943e-de3645b440ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:12.516135 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2dd0b8-c8a7-41ec-943e-de3645b440ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:12.517620 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2dd0b8-c8a7-41ec-943e-de3645b440ae None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:12.521204 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bc9030-444d-42d1-bc14-0dfd3c436eb1 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:12.530667 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 285/584] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 13:56:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:12.531515 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2dd0b8-c8a7-41ec-943e-de3645b440ae None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:12.560254 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698262a5-1165-48de-98da-8234524437f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:12.560882 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698262a5-1165-48de-98da-8234524437f7 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:12.561360 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698262a5-1165-48de-98da-8234524437f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:12.561844 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698262a5-1165-48de-98da-8234524437f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:12.573089 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698262a5-1165-48de-98da-8234524437f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:12.574520 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698262a5-1165-48de-98da-8234524437f7 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:12.589544 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-698262a5-1165-48de-98da-8234524437f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:12.589544 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:12.589544 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:12.589544 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:12.589544 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:12.589544 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:12.589544 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:12.589544 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:12.589544 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:12.589544 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:12.589544 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:12.589544 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:12.589544 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:12.589544 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:12.589544 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:12.590584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:12.590584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:12.590584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:12.590584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:12.590584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:12.590584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:12.590584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:12.590584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:12.590584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:12.590584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:12.590584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:12.590584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:12.590584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:12.590584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:12.590584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:12.590584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:12.590584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:12.591662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:12.591662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:12.591662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:12.591662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:12.591662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 300/585] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:12.591662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698262a5-1165-48de-98da-8234524437f7 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:12.636994 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2dd0b8-c8a7-41ec-943e-de3645b440ae None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:12.650490 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:12.676981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd397e-4d8c-4aa9-affe-d5b34c80197b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:12.677623 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd397e-4d8c-4aa9-affe-d5b34c80197b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:12.677900 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd397e-4d8c-4aa9-affe-d5b34c80197b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:12.678170 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd397e-4d8c-4aa9-affe-d5b34c80197b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:12.678955 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afd397e-4d8c-4aa9-affe-d5b34c80197b None admin] RBAC: Adding query filter params (role.id=d3d8169e71454005a8cf0d7544ad05dc, scope.project.id=72ecec9fad964b1fbeaefd0519597229, user.id=2fe2a6d7bd10435ba2c8e552d808b7a0) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:12.679166 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:12.679287 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afd397e-4d8c-4aa9-affe-d5b34c80197b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:12.681227 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afd397e-4d8c-4aa9-affe-d5b34c80197b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:12.686815 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 286/586] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:56:12 2022] GET /identity/v3/role_assignments?user.id=2fe2a6d7bd10435ba2c8e552d808b7a0&scope.project.id=72ecec9fad964b1fbeaefd0519597229&role.id=d3d8169e71454005a8cf0d7544ad05dc => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:12.687435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd397e-4d8c-4aa9-affe-d5b34c80197b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:12.698484 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698262a5-1165-48de-98da-8234524437f7 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:12.741050 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984fed7-7ae5-4512-bdbd-2976b1c860c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:12.741694 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984fed7-7ae5-4512-bdbd-2976b1c860c9 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:12.741976 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984fed7-7ae5-4512-bdbd-2976b1c860c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:12.742312 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984fed7-7ae5-4512-bdbd-2976b1c860c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:12.743106 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2984fed7-7ae5-4512-bdbd-2976b1c860c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:12.743351 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2984fed7-7ae5-4512-bdbd-2976b1c860c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:12.744795 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2984fed7-7ae5-4512-bdbd-2976b1c860c9 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:12.758783 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 301/587] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:12.759639 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984fed7-7ae5-4512-bdbd-2976b1c860c9 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:12.811444 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd397e-4d8c-4aa9-affe-d5b34c80197b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:12.854693 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971f5c46-528a-4602-9798-7f803f2482f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:12.855338 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971f5c46-528a-4602-9798-7f803f2482f2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:12.855626 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971f5c46-528a-4602-9798-7f803f2482f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:12.855907 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971f5c46-528a-4602-9798-7f803f2482f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:12.856655 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971f5c46-528a-4602-9798-7f803f2482f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:12.856967 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971f5c46-528a-4602-9798-7f803f2482f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:12.858336 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971f5c46-528a-4602-9798-7f803f2482f2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:12.875457 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 287/588] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:12.875457 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971f5c46-528a-4602-9798-7f803f2482f2 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:12.882562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984fed7-7ae5-4512-bdbd-2976b1c860c9 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:12.913296 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:12.922314 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da756a-84a8-47a9-8c12-9a670b2a807a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:12.923016 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da756a-84a8-47a9-8c12-9a670b2a807a None admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:12.923291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da756a-84a8-47a9-8c12-9a670b2a807a None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:12.923570 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da756a-84a8-47a9-8c12-9a670b2a807a None admin] PATH_INFO: `/v3/projects/72ecec9fad964b1fbeaefd0519597229/users/a246654c783b4cc68d212e91bb05d8c6/roles/d3d8169e71454005a8cf0d7544ad05dc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:12.939303 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79da756a-84a8-47a9-8c12-9a670b2a807a None admin] RBAC: Authorizing `identity:create_grant(project_id=72ecec9fad964b1fbeaefd0519597229, user_id=a246654c783b4cc68d212e91bb05d8c6, role_id=d3d8169e71454005a8cf0d7544ad05dc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:12.940783 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79da756a-84a8-47a9-8c12-9a670b2a807a None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:12.943105 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:12.960549 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 302/589] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 13:56:12 2022] PUT /identity/v3/projects/72ecec9fad964b1fbeaefd0519597229/users/a246654c783b4cc68d212e91bb05d8c6/roles/d3d8169e71454005a8cf0d7544ad05dc => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:56:12.960549 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da756a-84a8-47a9-8c12-9a670b2a807a None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:12.994005 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971f5c46-528a-4602-9798-7f803f2482f2 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:13.045971 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d61f506-621f-49aa-8113-2b3ab38ca0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:13.046791 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61f506-621f-49aa-8113-2b3ab38ca0dc None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:13.047087 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61f506-621f-49aa-8113-2b3ab38ca0dc None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:13.047451 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61f506-621f-49aa-8113-2b3ab38ca0dc None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:13.048186 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d61f506-621f-49aa-8113-2b3ab38ca0dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:13.048517 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d61f506-621f-49aa-8113-2b3ab38ca0dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:13.050763 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d61f506-621f-49aa-8113-2b3ab38ca0dc None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:13.069089 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 288/590] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:13.070145 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d61f506-621f-49aa-8113-2b3ab38ca0dc None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:13.101037 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da756a-84a8-47a9-8c12-9a670b2a807a None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:13.149708 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626306f4-c531-4b59-a69f-3136bb875e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:13.150790 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626306f4-c531-4b59-a69f-3136bb875e30 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:13.151279 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626306f4-c531-4b59-a69f-3136bb875e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:13.151750 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626306f4-c531-4b59-a69f-3136bb875e30 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:13.159461 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626306f4-c531-4b59-a69f-3136bb875e30 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:13.161231 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626306f4-c531-4b59-a69f-3136bb875e30 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:13.173853 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-626306f4-c531-4b59-a69f-3136bb875e30 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 13:56:13.173853 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:13.173853 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:13.173853 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:13.173853 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:13.173853 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:13.173853 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:13.173853 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:13.173853 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:13.173853 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:13.173853 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:13.173853 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:13.173853 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:56:13.173853 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:56:13.173853 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:56:13.173853 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:56:13.174772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:13.174772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:13.174772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:56:13.174772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:13.174772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:56:13.174772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:13.174772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:13.174772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:13.174772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:13.174772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:13.174772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:13.174772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:13.174772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:13.174772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:13.174772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:13.174772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:13.174772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:13.174772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:13.175688 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:13.175688 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:13.175688 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:56:13.175688 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:56:13.175688 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:56:13.175688 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:56:13.175688 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:56:13.175688 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:56:13.175688 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 13:56:13.175688 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:13.178235 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 303/591] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:56:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:13.179229 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626306f4-c531-4b59-a69f-3136bb875e30 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:13.193297 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:13.198691 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d61f506-621f-49aa-8113-2b3ab38ca0dc None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:13.243547 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e30f6a0-f687-4e89-bdd0-a97da32d3c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:13.244210 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30f6a0-f687-4e89-bdd0-a97da32d3c39 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:13.244513 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30f6a0-f687-4e89-bdd0-a97da32d3c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:13.244805 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30f6a0-f687-4e89-bdd0-a97da32d3c39 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:13.254371 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e30f6a0-f687-4e89-bdd0-a97da32d3c39 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:13.255940 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:13.256785 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e30f6a0-f687-4e89-bdd0-a97da32d3c39 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:13.276330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-5e30f6a0-f687-4e89-bdd0-a97da32d3c39 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:13.276330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:13.276330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:13.276330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:13.276330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:13.276330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:13.276330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:13.276330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:13.276330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:13.276330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:13.276330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:13.276330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:13.276330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:56:13.276330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:56:13.276330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:56:13.277283 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:56:13.277283 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:13.277283 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:13.277283 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:13.277283 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:13.277283 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:13.277283 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:13.277283 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:13.277283 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:13.277283 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:13.277283 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:13.277283 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:13.277283 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:13.277283 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:13.277283 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:13.277283 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:13.277283 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:13.278146 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:56:13.278146 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:56:13.278146 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:56:13.278146 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:56:13.278146 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:56:13.278146 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:56:13.278146 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:13.278146 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:13.278146 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 289/592] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:56:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:13.278647 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e30f6a0-f687-4e89-bdd0-a97da32d3c39 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:13.325921 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626306f4-c531-4b59-a69f-3136bb875e30 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:13.370422 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1165cb10-e354-4430-b2e3-2abdd1297e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:13.371080 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165cb10-e354-4430-b2e3-2abdd1297e15 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:13.372527 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165cb10-e354-4430-b2e3-2abdd1297e15 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:13.372527 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165cb10-e354-4430-b2e3-2abdd1297e15 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:13.382391 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1165cb10-e354-4430-b2e3-2abdd1297e15 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:13.383872 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1165cb10-e354-4430-b2e3-2abdd1297e15 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:13.395527 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-1165cb10-e354-4430-b2e3-2abdd1297e15 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 13:56:13.395527 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:13.395527 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:13.395527 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:13.395527 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:13.395527 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:13.395527 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:13.395527 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:13.395527 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:13.395527 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:13.395527 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:13.395527 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:13.395527 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:56:13.395527 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:56:13.395527 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:56:13.395527 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:56:13.396465 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:13.396465 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:13.396465 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:56:13.396465 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:13.396465 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:56:13.396465 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:13.396465 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:13.396465 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:13.396465 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:13.396465 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:13.396465 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:13.396465 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:13.396465 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:13.396465 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:13.396465 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:13.396465 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:13.396465 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:13.396465 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:13.397523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:13.397523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:13.397523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:56:13.397523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:56:13.397523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:56:13.397523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:56:13.397523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:56:13.397523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:56:13.397523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 13:56:13.397523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:13.397523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 304/593] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 13:56:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:13.398097 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1165cb10-e354-4430-b2e3-2abdd1297e15 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:13.422253 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e30f6a0-f687-4e89-bdd0-a97da32d3c39 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:13.472905 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9c2ea-96c4-4c8f-8763-e5d1358181e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:13.473886 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9c2ea-96c4-4c8f-8763-e5d1358181e7 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:13.474279 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9c2ea-96c4-4c8f-8763-e5d1358181e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:13.474678 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9c2ea-96c4-4c8f-8763-e5d1358181e7 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:13.477332 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f9c2ea-96c4-4c8f-8763-e5d1358181e7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:13.477667 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f9c2ea-96c4-4c8f-8763-e5d1358181e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:13.479100 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f9c2ea-96c4-4c8f-8763-e5d1358181e7 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:13.497660 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 290/594] 10.222.0.48 () {56 vars in 1137 bytes} [Mon Aug 1 13:56:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:13.498556 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9c2ea-96c4-4c8f-8763-e5d1358181e7 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:13.516359 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1165cb10-e354-4430-b2e3-2abdd1297e15 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:13.526215 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:13.533295 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:13.564301 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b374ca-f3bb-423c-bdf2-b3a9cf6dd793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:13.565227 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b374ca-f3bb-423c-bdf2-b3a9cf6dd793 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:13.565696 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b374ca-f3bb-423c-bdf2-b3a9cf6dd793 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:13.566210 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b374ca-f3bb-423c-bdf2-b3a9cf6dd793 None admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:13.567171 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b374ca-f3bb-423c-bdf2-b3a9cf6dd793 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:13.567738 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b374ca-f3bb-423c-bdf2-b3a9cf6dd793 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:13.569384 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b374ca-f3bb-423c-bdf2-b3a9cf6dd793 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:13.588419 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 305/595] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 13:56:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:13.589717 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b374ca-f3bb-423c-bdf2-b3a9cf6dd793 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:13.624424 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9c2ea-96c4-4c8f-8763-e5d1358181e7 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:13.664418 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1441a04-47f6-4797-8b65-b69297cfb023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:13.665167 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1441a04-47f6-4797-8b65-b69297cfb023 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:13.665511 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1441a04-47f6-4797-8b65-b69297cfb023 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:13.665855 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1441a04-47f6-4797-8b65-b69297cfb023 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:13.666875 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1441a04-47f6-4797-8b65-b69297cfb023 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:13.668053 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1441a04-47f6-4797-8b65-b69297cfb023 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:13.669579 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1441a04-47f6-4797-8b65-b69297cfb023 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:13.692529 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 291/596] 10.222.0.48 () {56 vars in 1139 bytes} [Mon Aug 1 13:56:13 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:13.693175 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1441a04-47f6-4797-8b65-b69297cfb023 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:13.708677 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b374ca-f3bb-423c-bdf2-b3a9cf6dd793 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:13.747321 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e654d-4508-4d31-a7c4-547229d2151d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:13.748054 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e654d-4508-4d31-a7c4-547229d2151d None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:13.748345 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e654d-4508-4d31-a7c4-547229d2151d None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:13.748650 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e654d-4508-4d31-a7c4-547229d2151d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:13.762101 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e654d-4508-4d31-a7c4-547229d2151d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:13.763643 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e654d-4508-4d31-a7c4-547229d2151d None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:13.780743 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-aa3e654d-4508-4d31-a7c4-547229d2151d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:13.780743 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:13.780743 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:13.780743 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:13.780743 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:13.780743 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:13.780743 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:13.780743 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:13.780743 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:13.780743 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:13.780743 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:13.780743 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:13.780743 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:13.780743 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:13.780743 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:13.781614 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:13.781614 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:13.781614 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:13.781614 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:13.781614 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:13.781614 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:13.781614 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:13.781614 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:13.781614 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:13.781614 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:13.781614 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:13.781614 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:13.781614 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:13.781614 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:13.781614 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:13.781614 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:13.781614 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:13.782452 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:13.782452 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:13.782452 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:13.782452 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:13.786871 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 306/597] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:13.787745 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e654d-4508-4d31-a7c4-547229d2151d None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:13.794080 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:13.829387 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1441a04-47f6-4797-8b65-b69297cfb023 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:13.830547 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:13.875934 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d60a90b-01e3-41cb-ac84-b193a1070087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:13.877214 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60a90b-01e3-41cb-ac84-b193a1070087 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:13.877644 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60a90b-01e3-41cb-ac84-b193a1070087 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:13.878091 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60a90b-01e3-41cb-ac84-b193a1070087 None admin] PATH_INFO: `/v3/projects/72ecec9fad964b1fbeaefd0519597229/users/b185b052f9df4d3f9847d8f9b4cfda5f/roles/d3d8169e71454005a8cf0d7544ad05dc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:13.896161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d60a90b-01e3-41cb-ac84-b193a1070087 None admin] RBAC: Authorizing `identity:create_grant(project_id=72ecec9fad964b1fbeaefd0519597229, user_id=b185b052f9df4d3f9847d8f9b4cfda5f, role_id=d3d8169e71454005a8cf0d7544ad05dc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:13.898147 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d60a90b-01e3-41cb-ac84-b193a1070087 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:13.920575 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 292/598] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 13:56:13 2022] PUT /identity/v3/projects/72ecec9fad964b1fbeaefd0519597229/users/b185b052f9df4d3f9847d8f9b4cfda5f/roles/d3d8169e71454005a8cf0d7544ad05dc => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:56:13.922461 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d60a90b-01e3-41cb-ac84-b193a1070087 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:13.944073 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e654d-4508-4d31-a7c4-547229d2151d None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:14.027957 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1372a9-ac48-4a80-98d4-973e1b755a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:14.029159 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1372a9-ac48-4a80-98d4-973e1b755a73 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:14.029650 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1372a9-ac48-4a80-98d4-973e1b755a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:14.030126 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1372a9-ac48-4a80-98d4-973e1b755a73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:14.044325 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1372a9-ac48-4a80-98d4-973e1b755a73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:14.046652 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1372a9-ac48-4a80-98d4-973e1b755a73 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:14.064131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-ae1372a9-ac48-4a80-98d4-973e1b755a73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:14.064131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:14.064131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:14.064131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:14.064131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:14.064131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:14.064131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:14.064131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:14.064131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:14.064131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:14.064131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:14.064131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:14.064131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:14.064131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:14.064131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:14.065104 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:14.065104 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:14.065104 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:14.065104 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:14.065104 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:14.065104 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:14.065104 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:14.065104 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:14.065104 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:14.065104 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:14.065104 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:14.065104 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:14.065104 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:14.065104 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:14.065104 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:14.065104 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:14.065104 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:14.065958 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:14.065958 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:14.065958 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:14.065958 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:14.069066 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 307/599] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:14.077540 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1372a9-ac48-4a80-98d4-973e1b755a73 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:14.152309 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d60a90b-01e3-41cb-ac84-b193a1070087 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:14.188848 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:14.200973 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01fe08-76f9-45fb-a24d-b0466241b8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:14.201532 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01fe08-76f9-45fb-a24d-b0466241b8d7 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:14.201930 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01fe08-76f9-45fb-a24d-b0466241b8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:14.202223 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01fe08-76f9-45fb-a24d-b0466241b8d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:14.202943 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c01fe08-76f9-45fb-a24d-b0466241b8d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:14.203253 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c01fe08-76f9-45fb-a24d-b0466241b8d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:14.203758 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:14.204883 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c01fe08-76f9-45fb-a24d-b0466241b8d7 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:14.222369 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 293/600] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:14.223777 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01fe08-76f9-45fb-a24d-b0466241b8d7 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:14.231701 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1372a9-ac48-4a80-98d4-973e1b755a73 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:14.276061 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf61b97-8812-432e-ab96-4459bd14d369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:14.277323 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf61b97-8812-432e-ab96-4459bd14d369 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:14.277765 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf61b97-8812-432e-ab96-4459bd14d369 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:14.278431 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf61b97-8812-432e-ab96-4459bd14d369 None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:14.279578 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf61b97-8812-432e-ab96-4459bd14d369 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:14.280029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf61b97-8812-432e-ab96-4459bd14d369 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:14.281719 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf61b97-8812-432e-ab96-4459bd14d369 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:14.299854 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 308/601] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:14.306948 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf61b97-8812-432e-ab96-4459bd14d369 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:14.354926 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01fe08-76f9-45fb-a24d-b0466241b8d7 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:14.420980 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf61b97-8812-432e-ab96-4459bd14d369 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:14.449787 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e44c4a0-b10f-45ba-b0f5-538474cca18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:14.450608 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44c4a0-b10f-45ba-b0f5-538474cca18e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:14.450996 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44c4a0-b10f-45ba-b0f5-538474cca18e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:14.451419 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44c4a0-b10f-45ba-b0f5-538474cca18e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:14.458836 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae58307-cdd9-4a7c-861b-caf562f326ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:14.459707 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae58307-cdd9-4a7c-861b-caf562f326ed None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:14.460117 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae58307-cdd9-4a7c-861b-caf562f326ed None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:14.460559 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae58307-cdd9-4a7c-861b-caf562f326ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:14.464572 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e44c4a0-b10f-45ba-b0f5-538474cca18e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:14.466200 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e44c4a0-b10f-45ba-b0f5-538474cca18e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:14.468902 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae58307-cdd9-4a7c-861b-caf562f326ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:14.470484 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae58307-cdd9-4a7c-861b-caf562f326ed None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:14.472804 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:14.483946 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-bae58307-cdd9-4a7c-861b-caf562f326ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:14.483946 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:14.483946 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:14.483946 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:14.483946 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:14.483946 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:14.483946 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:14.483946 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:14.483946 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:14.483946 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:14.483946 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:14.483946 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:14.483946 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:56:14.483946 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:56:14.483946 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:56:14.485112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:56:14.485112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:14.485112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:14.485112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:14.485112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:14.485112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:14.485112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:14.485112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:14.485112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:14.485112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:14.485112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:14.485112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:14.485112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:14.485112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:14.485112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:14.485112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:14.485112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:14.486138 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:56:14.486138 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:56:14.486138 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:56:14.486138 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:56:14.486138 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:56:14.486138 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:56:14.486138 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:14.486138 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:14.488149 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 309/602] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:56:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:14.491596 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-6e44c4a0-b10f-45ba-b0f5-538474cca18e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:14.491596 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:14.491596 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:14.491596 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:14.491596 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:14.491596 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:14.491596 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:14.491596 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:14.491596 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:14.491596 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:14.491596 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:14.491596 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:14.491596 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:56:14.491596 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:56:14.491596 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:56:14.492665 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:56:14.492665 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:14.492665 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:14.492665 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:14.492665 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:14.492665 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:14.492665 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:14.492665 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:14.492665 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:14.492665 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:14.492665 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:14.492665 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:14.492665 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:14.492665 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:14.492665 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:14.492665 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:14.492665 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:14.493741 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:56:14.493741 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:56:14.493741 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:56:14.493741 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:56:14.493741 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:56:14.493741 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:56:14.493741 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:14.493741 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:14.495567 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 294/603] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:56:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:14.496646 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae58307-cdd9-4a7c-861b-caf562f326ed None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:14.500761 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e44c4a0-b10f-45ba-b0f5-538474cca18e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:14.587363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:14.689598 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae58307-cdd9-4a7c-861b-caf562f326ed None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:14.703886 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e44c4a0-b10f-45ba-b0f5-538474cca18e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:14.737231 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee78213-c184-4618-8f45-3de2377c78ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:14.737859 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee78213-c184-4618-8f45-3de2377c78ed None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:14.738141 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee78213-c184-4618-8f45-3de2377c78ed None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:14.738423 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee78213-c184-4618-8f45-3de2377c78ed None admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:14.739263 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee78213-c184-4618-8f45-3de2377c78ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:14.739577 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee78213-c184-4618-8f45-3de2377c78ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:14.741033 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee78213-c184-4618-8f45-3de2377c78ed None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:14.747647 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8021e6c5-2775-44c0-ba1f-da176a6166fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:14.748278 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8021e6c5-2775-44c0-ba1f-da176a6166fc None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:14.748579 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8021e6c5-2775-44c0-ba1f-da176a6166fc None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:14.748854 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8021e6c5-2775-44c0-ba1f-da176a6166fc None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:14.749611 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8021e6c5-2775-44c0-ba1f-da176a6166fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:14.749916 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8021e6c5-2775-44c0-ba1f-da176a6166fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:14.751344 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8021e6c5-2775-44c0-ba1f-da176a6166fc None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:14.754207 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 310/604] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 13:56:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:14.760943 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee78213-c184-4618-8f45-3de2377c78ed None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:14.765583 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 295/605] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 13:56:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:56:14.777498 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8021e6c5-2775-44c0-ba1f-da176a6166fc None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:14.834577 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:14.871067 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee78213-c184-4618-8f45-3de2377c78ed None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:14.909018 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f3d33-a186-415b-99d1-02c4ab80867b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:14.909931 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f3d33-a186-415b-99d1-02c4ab80867b None admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:14.910370 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f3d33-a186-415b-99d1-02c4ab80867b None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:14.910649 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f3d33-a186-415b-99d1-02c4ab80867b None admin] PATH_INFO: `/v3/projects/72ecec9fad964b1fbeaefd0519597229/users/fb9f698e09d246bcbea855613764fbe1/roles/d3d8169e71454005a8cf0d7544ad05dc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:14.925652 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f3d33-a186-415b-99d1-02c4ab80867b None admin] RBAC: Authorizing `identity:create_grant(project_id=72ecec9fad964b1fbeaefd0519597229, user_id=fb9f698e09d246bcbea855613764fbe1, role_id=d3d8169e71454005a8cf0d7544ad05dc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:14.927099 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f3d33-a186-415b-99d1-02c4ab80867b None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:14.944198 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8021e6c5-2775-44c0-ba1f-da176a6166fc None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:14.948699 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:14.950675 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 311/606] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 13:56:14 2022] PUT /identity/v3/projects/72ecec9fad964b1fbeaefd0519597229/users/fb9f698e09d246bcbea855613764fbe1/roles/d3d8169e71454005a8cf0d7544ad05dc => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 13:56:14.957778 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62ab6b-5267-4e03-a1d2-cd4e9618b57d None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:14.957778 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62ab6b-5267-4e03-a1d2-cd4e9618b57d None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:14.957778 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62ab6b-5267-4e03-a1d2-cd4e9618b57d None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:14.960538 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 312/607] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:14 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:14.965541 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982047ce-f050-4466-aa76-2d1642d9236d None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:14.965859 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982047ce-f050-4466-aa76-2d1642d9236d None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:14.966090 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982047ce-f050-4466-aa76-2d1642d9236d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:15.000129 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-982047ce-f050-4466-aa76-2d1642d9236d None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:15.004261 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-982047ce-f050-4466-aa76-2d1642d9236d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:15.022057 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f9c002-6e54-40a9-a8fc-e797937910eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:15.022754 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9c002-6e54-40a9-a8fc-e797937910eb None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:15.023033 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9c002-6e54-40a9-a8fc-e797937910eb None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:15.023847 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9c002-6e54-40a9-a8fc-e797937910eb None admin] PATH_INFO: `/v3/projects/72ecec9fad964b1fbeaefd0519597229/users/39a66f7362964389be039937a786bb0a/roles/d3d8169e71454005a8cf0d7544ad05dc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:15.050656 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f9c002-6e54-40a9-a8fc-e797937910eb None admin] RBAC: Authorizing `identity:create_grant(project_id=72ecec9fad964b1fbeaefd0519597229, user_id=39a66f7362964389be039937a786bb0a, role_id=d3d8169e71454005a8cf0d7544ad05dc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:15.052128 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f9c002-6e54-40a9-a8fc-e797937910eb None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:15.071955 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 296/608] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 13:56:14 2022] PUT /identity/v3/projects/72ecec9fad964b1fbeaefd0519597229/users/39a66f7362964389be039937a786bb0a/roles/d3d8169e71454005a8cf0d7544ad05dc => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:56:15.157381 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 313/609] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:56:15.182698 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f9c002-6e54-40a9-a8fc-e797937910eb None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:15.258714 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c4f5e-9322-438d-ab80-a466a2aedd0c None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:15.258982 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c4f5e-9322-438d-ab80-a466a2aedd0c None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:15.259258 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c4f5e-9322-438d-ab80-a466a2aedd0c None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:15.260651 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 314/610] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:56:15.274077 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c4bd58-0bce-4ce5-9e1a-b616c9061828 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:15.274414 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c4bd58-0bce-4ce5-9e1a-b616c9061828 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:15.274638 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c4bd58-0bce-4ce5-9e1a-b616c9061828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:15.298056 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-46c4bd58-0bce-4ce5-9e1a-b616c9061828 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:15.299573 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-46c4bd58-0bce-4ce5-9e1a-b616c9061828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:15.319982 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f9c002-6e54-40a9-a8fc-e797937910eb None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:15.338934 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:15.370452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd2748-3c18-453d-8e19-d054ae2f26a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:15.371595 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd2748-3c18-453d-8e19-d054ae2f26a8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:15.372087 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd2748-3c18-453d-8e19-d054ae2f26a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:15.372617 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd2748-3c18-453d-8e19-d054ae2f26a8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:15.387137 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 315/611] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:56:15.390140 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd2748-3c18-453d-8e19-d054ae2f26a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:56:15.395372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-34cd2748-3c18-453d-8e19-d054ae2f26a8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:56:15.395372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:15.395372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:15.395372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:15.395372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:15.395372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:15.395372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:15.395372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:15.395372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:15.395372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:15.395372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:15.395372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:15.395372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:56:15.395372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:56:15.395372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:56:15.395372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:56:15.396544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:56:15.396544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:56:15.396544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:56:15.396544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:56:15.396544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:15.396544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:15.396544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:15.396544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:15.396544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:15.396544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:15.396544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:15.396544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:15.396544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:15.396544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:15.396544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:15.396544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:15.397587 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:15.397587 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:15.397587 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:15.397587 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:15.397587 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:56:15.397587 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:56:15.397587 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:56:15.397587 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:56:15.397587 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:56:15.397587 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:56:15.397587 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:56:15.397587 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:15.399890 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 297/612] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 13:56:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:15.405946 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c4bd58-0bce-4ce5-9e1a-b616c9061828 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:15.426385 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd2748-3c18-453d-8e19-d054ae2f26a8 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:15.451209 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:15.600358 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c4bd58-0bce-4ce5-9e1a-b616c9061828 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:15.613732 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd2748-3c18-453d-8e19-d054ae2f26a8 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:15.637931 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34038056-1b4a-4746-adcf-a2b660996c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:15.638697 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34038056-1b4a-4746-adcf-a2b660996c93 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:15.639286 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34038056-1b4a-4746-adcf-a2b660996c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:15.639657 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34038056-1b4a-4746-adcf-a2b660996c93 None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:15.640574 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34038056-1b4a-4746-adcf-a2b660996c93 None admin] RBAC: Adding query filter params (name=service) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:15.640981 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34038056-1b4a-4746-adcf-a2b660996c93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:15.642442 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34038056-1b4a-4746-adcf-a2b660996c93 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:15.651690 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 316/613] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 13:56:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:15.657608 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34038056-1b4a-4746-adcf-a2b660996c93 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:15.686716 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0c014-efdf-4dc1-a57c-5c07accc0517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:15.687704 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0c014-efdf-4dc1-a57c-5c07accc0517 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:15.691107 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0c014-efdf-4dc1-a57c-5c07accc0517 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:15.691587 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0c014-efdf-4dc1-a57c-5c07accc0517 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:15.712298 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef0c014-efdf-4dc1-a57c-5c07accc0517 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:56:15.713950 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:15.718362 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-fef0c014-efdf-4dc1-a57c-5c07accc0517 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:56:15.718362 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:15.718362 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:15.718362 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:15.718362 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:15.718362 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:15.718362 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:15.718362 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:15.718362 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:15.718362 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:15.718362 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:15.718362 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:15.718362 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:56:15.718362 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:56:15.718362 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:56:15.718362 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:56:15.719300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:56:15.719300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:56:15.719300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:56:15.719300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:56:15.719300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:15.719300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:15.719300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:15.719300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:15.719300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:15.719300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:15.719300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:15.719300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:15.719300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:15.719300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:15.719300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:15.719300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:15.720147 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:15.720147 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:15.720147 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:15.720147 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:15.720147 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:56:15.720147 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:56:15.720147 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:56:15.720147 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:56:15.720147 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:56:15.720147 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:56:15.720147 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:56:15.720147 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:15.722900 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 298/614] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 13:56:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:56:15.728838 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0c014-efdf-4dc1-a57c-5c07accc0517 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:15.865501 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:15.885898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34038056-1b4a-4746-adcf-a2b660996c93 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:15.910477 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0c014-efdf-4dc1-a57c-5c07accc0517 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:15.951596 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e7ec4-59e9-487f-adf4-47ba1bea5628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:15.952171 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e7ec4-59e9-487f-adf4-47ba1bea5628 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:15.952454 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e7ec4-59e9-487f-adf4-47ba1bea5628 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:15.952755 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e7ec4-59e9-487f-adf4-47ba1bea5628 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:15.973783 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710e7ec4-59e9-487f-adf4-47ba1bea5628 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:15.975278 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710e7ec4-59e9-487f-adf4-47ba1bea5628 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:15.984287 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff3fb9-801e-4306-a9ea-522f3924c94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:15.984943 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff3fb9-801e-4306-a9ea-522f3924c94d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:15.985225 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff3fb9-801e-4306-a9ea-522f3924c94d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:15.985501 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff3fb9-801e-4306-a9ea-522f3924c94d None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:15.986232 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cff3fb9-801e-4306-a9ea-522f3924c94d None admin] RBAC: Adding query filter params (name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:15.986537 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cff3fb9-801e-4306-a9ea-522f3924c94d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:15.987896 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cff3fb9-801e-4306-a9ea-522f3924c94d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:16.002436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-710e7ec4-59e9-487f-adf4-47ba1bea5628 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:16.002436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:16.002436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:16.002436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:16.002436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:16.002436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:16.002436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:16.002436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:16.002436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:16.002436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:16.002436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:16.002436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:16.002436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:16.002436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:16.002436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:16.003389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:16.003389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:16.003389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:16.003389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:16.003389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:16.003389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:16.003389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:16.003389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:16.003389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:16.003389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:16.003389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:16.003389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:16.003389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:16.003389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:16.003389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:16.003389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:16.003389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:16.004263 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:16.004263 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:16.004263 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:16.004263 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:16.005856 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 299/615] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 13:56:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:56:16.006108 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 317/616] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:16.014842 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff3fb9-801e-4306-a9ea-522f3924c94d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:16.016113 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e7ec4-59e9-487f-adf4-47ba1bea5628 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:16.172835 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:16.205227 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff3fb9-801e-4306-a9ea-522f3924c94d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:16.206434 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e7ec4-59e9-487f-adf4-47ba1bea5628 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:16.250396 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0803508-5595-443b-b61f-ee9a7807ec1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:16.251040 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0803508-5595-443b-b61f-ee9a7807ec1f None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:16.251320 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0803508-5595-443b-b61f-ee9a7807ec1f None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:16.251595 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0803508-5595-443b-b61f-ee9a7807ec1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:16.254076 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4effcb-a345-487d-8e8d-3df92a517325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:16.254693 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4effcb-a345-487d-8e8d-3df92a517325 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:16.254970 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4effcb-a345-487d-8e8d-3df92a517325 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:16.255241 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4effcb-a345-487d-8e8d-3df92a517325 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:16.255937 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4effcb-a345-487d-8e8d-3df92a517325 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:16.256508 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4effcb-a345-487d-8e8d-3df92a517325 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:16.261340 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4effcb-a345-487d-8e8d-3df92a517325 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:16.264424 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:16.271191 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0803508-5595-443b-b61f-ee9a7807ec1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:16.274778 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0803508-5595-443b-b61f-ee9a7807ec1f None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:16.276131 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 300/617] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:16.277420 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627d184-bf14-4447-9794-beedf2698f25 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:16.277710 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627d184-bf14-4447-9794-beedf2698f25 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:16.277988 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627d184-bf14-4447-9794-beedf2698f25 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:16.283244 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 301/618] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:16.284033 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627d184-bf14-4447-9794-beedf2698f25 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:16.294135 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-d0803508-5595-443b-b61f-ee9a7807ec1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:16.294135 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:16.294135 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:16.294135 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:16.294135 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:16.294135 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:16.294135 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:16.294135 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:16.294135 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:16.294135 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:16.294135 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:16.294135 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:16.294135 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:16.294135 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:16.294135 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:16.295012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:16.295012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:16.295012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:16.295012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:16.295012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:16.295012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:16.295012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:16.295012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:16.295012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:16.295012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:16.295012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:16.295012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:16.295012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:16.295012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:16.295012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:16.295012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:16.295012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:16.295863 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:16.295863 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:16.295863 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:16.295863 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:16.298140 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 318/619] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:16.299435 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4182c3-cc1b-4845-961f-a03f856ad46a None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:16.299936 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4182c3-cc1b-4845-961f-a03f856ad46a None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:16.300339 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4182c3-cc1b-4845-961f-a03f856ad46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:16.325763 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-4c4182c3-cc1b-4845-961f-a03f856ad46a None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:16.326962 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-4c4182c3-cc1b-4845-961f-a03f856ad46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:16.403811 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627d184-bf14-4447-9794-beedf2698f25 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:16.407164 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 319/620] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:56:16.408251 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4182c3-cc1b-4845-961f-a03f856ad46a None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:16.464112 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c7a23-e1b5-47ab-8f57-e5be8870f5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:16.464874 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c7a23-e1b5-47ab-8f57-e5be8870f5a5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:16.465143 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c7a23-e1b5-47ab-8f57-e5be8870f5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:16.465424 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c7a23-e1b5-47ab-8f57-e5be8870f5a5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:16.473157 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c7a23-e1b5-47ab-8f57-e5be8870f5a5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:16.474762 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c7a23-e1b5-47ab-8f57-e5be8870f5a5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:16.487079 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-576c7a23-e1b5-47ab-8f57-e5be8870f5a5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 13:56:16.487079 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:16.487079 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:16.487079 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:16.487079 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:16.487079 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:16.487079 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:16.487079 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:16.487079 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:16.487079 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:16.487079 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:16.487079 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:16.487079 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:56:16.487079 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:56:16.487079 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:56:16.487079 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:56:16.488196 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:16.488196 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:16.488196 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:56:16.488196 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:16.488196 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:56:16.488196 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:16.488196 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:16.488196 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:16.488196 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:16.488196 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:16.488196 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:16.488196 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:16.488196 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:16.488196 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:16.488196 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:16.488196 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:16.488196 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:16.488196 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:16.489171 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:16.489171 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:16.489171 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:56:16.489171 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:56:16.489171 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:56:16.489171 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:56:16.489171 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:56:16.489171 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:56:16.489171 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 13:56:16.489171 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:16.492739 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 302/621] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 13:56:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:16.493577 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c7a23-e1b5-47ab-8f57-e5be8870f5a5 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:16.495932 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:16.546604 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4182c3-cc1b-4845-961f-a03f856ad46a None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:16.558237 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:16.592293 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f01b3f-5eda-41ed-b49b-dff38cec4e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:16.593683 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f01b3f-5eda-41ed-b49b-dff38cec4e2b None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:16.594305 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f01b3f-5eda-41ed-b49b-dff38cec4e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:16.594862 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f01b3f-5eda-41ed-b49b-dff38cec4e2b None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:16.595900 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f01b3f-5eda-41ed-b49b-dff38cec4e2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:16.596362 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f01b3f-5eda-41ed-b49b-dff38cec4e2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:16.598510 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f01b3f-5eda-41ed-b49b-dff38cec4e2b None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:16.608771 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c7a23-e1b5-47ab-8f57-e5be8870f5a5 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:16.615654 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 320/622] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:16.616678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f01b3f-5eda-41ed-b49b-dff38cec4e2b None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:16.648634 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d3617-bd11-4f44-99f4-a04d2ed96181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:16.649454 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d3617-bd11-4f44-99f4-a04d2ed96181 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:16.649840 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d3617-bd11-4f44-99f4-a04d2ed96181 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:16.650246 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d3617-bd11-4f44-99f4-a04d2ed96181 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:16.660946 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043d3617-bd11-4f44-99f4-a04d2ed96181 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:56:16.666932 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-043d3617-bd11-4f44-99f4-a04d2ed96181 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:56:16.666932 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:16.666932 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:16.666932 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:16.666932 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:16.666932 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:16.666932 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:16.666932 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:16.666932 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:16.666932 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:16.666932 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:16.666932 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:16.666932 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:56:16.666932 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:56:16.666932 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:56:16.666932 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:56:16.668145 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:56:16.668145 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:56:16.668145 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:56:16.668145 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:56:16.668145 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:16.668145 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:16.668145 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:16.668145 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:16.668145 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:16.668145 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:16.668145 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:16.668145 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:16.668145 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:16.668145 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:16.668145 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:16.668145 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:16.669234 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:16.669234 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:16.669234 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:16.669234 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:16.669234 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:56:16.669234 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:56:16.669234 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:56:16.669234 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:56:16.669234 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:56:16.669234 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:56:16.669234 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:56:16.669234 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:16.675824 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 303/623] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 13:56:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:16.676684 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d3617-bd11-4f44-99f4-a04d2ed96181 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:16.744056 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f01b3f-5eda-41ed-b49b-dff38cec4e2b None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:16.770903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:16.784377 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae23c10-b059-4ce0-8610-f8032de0f367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:16.785153 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae23c10-b059-4ce0-8610-f8032de0f367 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:16.785529 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae23c10-b059-4ce0-8610-f8032de0f367 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:16.785897 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae23c10-b059-4ce0-8610-f8032de0f367 None admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:16.787063 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae23c10-b059-4ce0-8610-f8032de0f367 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:16.787306 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae23c10-b059-4ce0-8610-f8032de0f367 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:16.789451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae23c10-b059-4ce0-8610-f8032de0f367 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:16.801230 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d3617-bd11-4f44-99f4-a04d2ed96181 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:16.820728 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 321/624] 10.222.0.48 () {56 vars in 1135 bytes} [Mon Aug 1 13:56:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:16.826002 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:16.827369 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae23c10-b059-4ce0-8610-f8032de0f367 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:16.846017 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7bfc0b-6313-435d-a0e2-02c698e94d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:16.847182 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7bfc0b-6313-435d-a0e2-02c698e94d70 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:16.847451 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7bfc0b-6313-435d-a0e2-02c698e94d70 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:16.847731 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7bfc0b-6313-435d-a0e2-02c698e94d70 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:16.858712 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7bfc0b-6313-435d-a0e2-02c698e94d70 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:16.860418 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7bfc0b-6313-435d-a0e2-02c698e94d70 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:16.876141 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-fe7bfc0b-6313-435d-a0e2-02c698e94d70 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 13:56:16.876141 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:16.876141 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:16.876141 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:16.876141 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:16.876141 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:16.876141 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:16.876141 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:16.876141 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:16.876141 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:16.876141 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:16.876141 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:16.876141 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:56:16.876141 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:56:16.876141 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:56:16.876141 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:56:16.877120 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:16.877120 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:16.877120 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:56:16.877120 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:16.877120 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:56:16.877120 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:16.877120 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:16.877120 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:16.877120 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:16.877120 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:16.877120 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:16.877120 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:16.877120 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:16.877120 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:16.877120 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:16.877120 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:16.877120 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:16.877120 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:16.878052 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:16.878052 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:16.878052 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:56:16.878052 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:56:16.878052 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:56:16.878052 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:56:16.878052 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:56:16.878052 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:56:16.878052 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 13:56:16.878052 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:16.878052 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 304/625] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Aug 1 13:56:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:16.878626 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7bfc0b-6313-435d-a0e2-02c698e94d70 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:16.945477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae23c10-b059-4ce0-8610-f8032de0f367 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:16.983641 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b7b4f-7332-4874-a7e8-e016ffaeb9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:16.985071 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b7b4f-7332-4874-a7e8-e016ffaeb9c3 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:16.985604 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b7b4f-7332-4874-a7e8-e016ffaeb9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:16.986161 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b7b4f-7332-4874-a7e8-e016ffaeb9c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:16.987181 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140b7b4f-7332-4874-a7e8-e016ffaeb9c3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:16.987690 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140b7b4f-7332-4874-a7e8-e016ffaeb9c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:16.989411 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140b7b4f-7332-4874-a7e8-e016ffaeb9c3 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:16.993339 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7bfc0b-6313-435d-a0e2-02c698e94d70 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:17.004185 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 322/626] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 13:56:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:17.005385 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b7b4f-7332-4874-a7e8-e016ffaeb9c3 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:17.029735 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1aa6c-123a-490b-8efc-99d63eedb945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:17.030158 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1aa6c-123a-490b-8efc-99d63eedb945 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:17.030448 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1aa6c-123a-490b-8efc-99d63eedb945 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:17.030731 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1aa6c-123a-490b-8efc-99d63eedb945 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:17.043434 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e1aa6c-123a-490b-8efc-99d63eedb945 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:17.044927 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e1aa6c-123a-490b-8efc-99d63eedb945 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:17.060286 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:17.062102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-76e1aa6c-123a-490b-8efc-99d63eedb945 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:17.062102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:17.062102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:17.062102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:17.062102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:17.062102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:17.062102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:17.062102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:17.062102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:17.062102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:17.062102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:17.062102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:17.062102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:17.062102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:17.062102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:17.068704 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:17.068704 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:17.068704 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:17.068704 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:17.068704 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:17.068704 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:17.068704 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:17.068704 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:17.068704 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:17.068704 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:17.068704 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:17.068704 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:17.068704 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:17.068704 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:17.068704 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:17.068704 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:17.068704 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:17.069695 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:17.069695 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:17.069695 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:17.069695 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:17.069695 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 305/627] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:17.069695 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1aa6c-123a-490b-8efc-99d63eedb945 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:17.099912 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:17.145830 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b7b4f-7332-4874-a7e8-e016ffaeb9c3 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:17.182601 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a83285-1fe7-4ad8-8c99-5ebede504696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:17.183424 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a83285-1fe7-4ad8-8c99-5ebede504696 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:17.183971 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a83285-1fe7-4ad8-8c99-5ebede504696 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:17.184214 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a83285-1fe7-4ad8-8c99-5ebede504696 None admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:17.185216 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a83285-1fe7-4ad8-8c99-5ebede504696 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:17.185654 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a83285-1fe7-4ad8-8c99-5ebede504696 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:17.187286 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a83285-1fe7-4ad8-8c99-5ebede504696 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:17.194670 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1aa6c-123a-490b-8efc-99d63eedb945 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:17.206929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 323/628] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Aug 1 13:56:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:17.208012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a83285-1fe7-4ad8-8c99-5ebede504696 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:17.240447 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55095b-8294-462a-b5a5-05189137d260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:17.241668 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55095b-8294-462a-b5a5-05189137d260 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:17.242127 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55095b-8294-462a-b5a5-05189137d260 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:17.242924 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55095b-8294-462a-b5a5-05189137d260 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:17.257803 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc55095b-8294-462a-b5a5-05189137d260 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:17.259742 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc55095b-8294-462a-b5a5-05189137d260 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:17.278342 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-cc55095b-8294-462a-b5a5-05189137d260 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:17.278342 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:17.278342 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:17.278342 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:17.278342 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:17.278342 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:17.278342 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:17.278342 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:17.278342 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:17.278342 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:17.278342 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:17.278342 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:17.278342 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:17.278342 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:17.278342 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:17.279386 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:17.279386 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:17.279386 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:17.279386 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:17.279386 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:17.279386 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:17.279386 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:17.279386 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:17.279386 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:17.279386 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:17.279386 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:17.279386 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:17.279386 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:17.279386 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:17.279386 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:17.279386 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:17.279386 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:17.280465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:17.280465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:17.280465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:17.280465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:17.286330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 306/629] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:17.286330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55095b-8294-462a-b5a5-05189137d260 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:17.345200 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a83285-1fe7-4ad8-8c99-5ebede504696 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:17.346988 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:17.376457 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:17.380955 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003cd64-d741-4622-9024-5be5353f0f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:17.382111 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003cd64-d741-4622-9024-5be5353f0f8e None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:17.382396 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003cd64-d741-4622-9024-5be5353f0f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:17.382698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003cd64-d741-4622-9024-5be5353f0f8e None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:17.383389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7003cd64-d741-4622-9024-5be5353f0f8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:17.383694 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7003cd64-d741-4622-9024-5be5353f0f8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:17.385090 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7003cd64-d741-4622-9024-5be5353f0f8e None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:17.396121 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 324/630] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:17.396903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003cd64-d741-4622-9024-5be5353f0f8e None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:17.397502 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55095b-8294-462a-b5a5-05189137d260 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:17.431842 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85be89e-e2bb-4bb8-8a5c-5f3ef2fe3aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:17.432460 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85be89e-e2bb-4bb8-8a5c-5f3ef2fe3aec None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:17.432826 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85be89e-e2bb-4bb8-8a5c-5f3ef2fe3aec None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:17.433151 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85be89e-e2bb-4bb8-8a5c-5f3ef2fe3aec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:17.497575 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003cd64-d741-4622-9024-5be5353f0f8e None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:17.499004 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85be89e-e2bb-4bb8-8a5c-5f3ef2fe3aec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:17.500452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85be89e-e2bb-4bb8-8a5c-5f3ef2fe3aec None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:17.530336 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3696932e-acee-4523-b9c0-859bfd29bab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:17.530882 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3696932e-acee-4523-b9c0-859bfd29bab6 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:17.531163 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3696932e-acee-4523-b9c0-859bfd29bab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:17.531438 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3696932e-acee-4523-b9c0-859bfd29bab6 None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:17.532123 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3696932e-acee-4523-b9c0-859bfd29bab6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:17.532426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3696932e-acee-4523-b9c0-859bfd29bab6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:17.533810 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3696932e-acee-4523-b9c0-859bfd29bab6 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:17.545805 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 325/631] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:17.547049 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0adaf46-8f05-4942-ab84-916119d57b91 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:17.547342 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0adaf46-8f05-4942-ab84-916119d57b91 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:17.547619 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0adaf46-8f05-4942-ab84-916119d57b91 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:17.549038 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 326/632] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:17.550173 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891b2b20-72b3-423f-9fee-2504b21263fb None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:17.550314 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891b2b20-72b3-423f-9fee-2504b21263fb None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:17.550592 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891b2b20-72b3-423f-9fee-2504b21263fb None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:17.551786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 327/633] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:17.552641 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891b2b20-72b3-423f-9fee-2504b21263fb None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:17.579577 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:17.585813 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:17.605940 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 307/634] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:56:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:56:17.606762 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85be89e-e2bb-4bb8-8a5c-5f3ef2fe3aec None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:17.658097 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891b2b20-72b3-423f-9fee-2504b21263fb None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:17.691522 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9b65a-f254-4a64-94e6-21102df37dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:17.692174 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9b65a-f254-4a64-94e6-21102df37dfc None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:17.692460 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9b65a-f254-4a64-94e6-21102df37dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:17.692773 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9b65a-f254-4a64-94e6-21102df37dfc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:17.699955 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c9b65a-f254-4a64-94e6-21102df37dfc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:17.701440 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c9b65a-f254-4a64-94e6-21102df37dfc None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:17.704138 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85be89e-e2bb-4bb8-8a5c-5f3ef2fe3aec None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:17.713211 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-48c9b65a-f254-4a64-94e6-21102df37dfc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:17.713211 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:17.713211 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:17.713211 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:17.713211 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:17.713211 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:17.713211 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:17.713211 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:17.713211 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:17.713211 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:17.713211 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:17.713211 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:17.713211 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:56:17.713211 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:56:17.713211 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:56:17.714244 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:56:17.714244 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:17.714244 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:17.714244 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:17.714244 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:17.714244 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:17.714244 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:17.714244 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:17.714244 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:17.714244 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:17.714244 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:17.714244 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:17.714244 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:17.714244 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:17.714244 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:17.714244 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:17.714244 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:17.715183 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:56:17.715183 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:56:17.715183 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:56:17.715183 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:56:17.715183 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:56:17.715183 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:56:17.715183 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:17.715183 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:17.715183 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 328/635] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:56:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:17.715707 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1f46f-630c-4bb3-bbe9-f275c89068aa None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:17.715784 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1f46f-630c-4bb3-bbe9-f275c89068aa None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:17.716128 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1f46f-630c-4bb3-bbe9-f275c89068aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:17.733015 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-0aa1f46f-630c-4bb3-bbe9-f275c89068aa None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:17.734196 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-0aa1f46f-630c-4bb3-bbe9-f275c89068aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:17.741894 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86036cd-3c59-4d48-a715-066d41c787f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:17.742539 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86036cd-3c59-4d48-a715-066d41c787f1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:17.742822 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86036cd-3c59-4d48-a715-066d41c787f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:17.743099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86036cd-3c59-4d48-a715-066d41c787f1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:17.750175 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86036cd-3c59-4d48-a715-066d41c787f1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:17.751620 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86036cd-3c59-4d48-a715-066d41c787f1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:17.763870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-a86036cd-3c59-4d48-a715-066d41c787f1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 13:56:17.763870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:17.763870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:17.763870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:17.763870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:17.763870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:17.763870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:17.763870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:17.763870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:17.763870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:17.763870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:17.763870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:17.763870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:56:17.763870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:56:17.763870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:56:17.763870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:56:17.764871 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:17.764871 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:17.764871 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:56:17.764871 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:17.764871 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:56:17.764871 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:17.764871 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:17.764871 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:17.764871 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:17.764871 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:17.764871 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:17.764871 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:17.764871 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:17.764871 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:17.764871 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:17.764871 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:17.764871 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:17.764871 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:17.765786 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:17.765786 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:17.765786 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:56:17.765786 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:56:17.765786 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:56:17.765786 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:56:17.765786 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:56:17.765786 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:56:17.765786 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 13:56:17.765786 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:17.765786 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 308/636] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:56:17 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:17.766407 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd1895-1f63-4a78-8f86-625f0aa6d5ba None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:17.766482 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd1895-1f63-4a78-8f86-625f0aa6d5ba None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:17.766919 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd1895-1f63-4a78-8f86-625f0aa6d5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:17.784630 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-fbfd1895-1f63-4a78-8f86-625f0aa6d5ba None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:17.785736 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fbfd1895-1f63-4a78-8f86-625f0aa6d5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:17.808000 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 329/637] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:56:17.808874 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1f46f-630c-4bb3-bbe9-f275c89068aa None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:17.823757 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:17.826697 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:17.859512 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 309/638] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:56:17.860350 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd1895-1f63-4a78-8f86-625f0aa6d5ba None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:17.916505 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1f46f-630c-4bb3-bbe9-f275c89068aa None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:17.950471 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c3813f-9f12-4807-9f39-d970a758544f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:17.951115 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3813f-9f12-4807-9f39-d970a758544f None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:17.951403 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3813f-9f12-4807-9f39-d970a758544f None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:17.951681 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3813f-9f12-4807-9f39-d970a758544f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:17.961747 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c3813f-9f12-4807-9f39-d970a758544f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:17.961941 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd1895-1f63-4a78-8f86-625f0aa6d5ba None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:17.963186 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c3813f-9f12-4807-9f39-d970a758544f None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:17.977673 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-37c3813f-9f12-4807-9f39-d970a758544f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:17.977673 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:17.977673 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:17.977673 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:17.977673 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:17.977673 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:17.977673 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:17.977673 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:17.977673 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:17.977673 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:17.977673 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:17.977673 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:17.977673 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:17.977673 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:17.977673 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:17.978678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:17.978678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:17.978678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:17.978678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:17.978678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:17.978678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:17.978678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:17.978678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:17.978678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:17.978678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:17.978678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:17.978678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:17.978678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:17.978678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:17.978678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:17.978678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:17.978678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:17.979549 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:17.979549 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:17.979549 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:17.979549 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:17.979549 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 330/639] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:17.979549 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c3813f-9f12-4807-9f39-d970a758544f None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:17.999752 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2084b3c-18e1-4603-9937-7b4eb655c5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:18.000378 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2084b3c-18e1-4603-9937-7b4eb655c5d9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:18.000689 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2084b3c-18e1-4603-9937-7b4eb655c5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:18.000969 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2084b3c-18e1-4603-9937-7b4eb655c5d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:18.001719 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2084b3c-18e1-4603-9937-7b4eb655c5d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:18.002029 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2084b3c-18e1-4603-9937-7b4eb655c5d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:18.003443 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2084b3c-18e1-4603-9937-7b4eb655c5d9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:18.015516 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 310/640] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 13:56:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:18.016289 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2084b3c-18e1-4603-9937-7b4eb655c5d9 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:18.046155 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:18.048901 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:18.088263 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c3813f-9f12-4807-9f39-d970a758544f None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:18.114821 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2084b3c-18e1-4603-9937-7b4eb655c5d9 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:18.121288 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a1da2b-37ed-48e4-8c27-84e200a83d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:18.121899 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1da2b-37ed-48e4-8c27-84e200a83d3c None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:18.122179 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1da2b-37ed-48e4-8c27-84e200a83d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:18.122452 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1da2b-37ed-48e4-8c27-84e200a83d3c None admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:18.123238 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a1da2b-37ed-48e4-8c27-84e200a83d3c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:18.123542 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a1da2b-37ed-48e4-8c27-84e200a83d3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:18.124983 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a1da2b-37ed-48e4-8c27-84e200a83d3c None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:18.142655 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 331/641] 10.222.0.48 () {56 vars in 1137 bytes} [Mon Aug 1 13:56:17 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:18.143499 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a1da2b-37ed-48e4-8c27-84e200a83d3c None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:18.149938 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f6d207-7a19-491a-99de-84a5b9457c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:18.150585 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6d207-7a19-491a-99de-84a5b9457c6f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:18.150872 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6d207-7a19-491a-99de-84a5b9457c6f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:18.151150 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6d207-7a19-491a-99de-84a5b9457c6f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:18.162230 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f6d207-7a19-491a-99de-84a5b9457c6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:56:18.167624 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-97f6d207-7a19-491a-99de-84a5b9457c6f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:56:18.167624 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:18.167624 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:18.167624 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:18.167624 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:18.167624 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:18.167624 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:18.167624 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:18.167624 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:18.167624 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:18.167624 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:18.167624 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:18.167624 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:56:18.167624 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:56:18.167624 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:56:18.167624 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:56:18.168669 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:56:18.168669 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:56:18.168669 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:56:18.168669 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:56:18.168669 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:18.168669 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:18.168669 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:18.168669 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:18.168669 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:18.168669 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:18.168669 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:18.168669 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:18.168669 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:18.168669 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:18.168669 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:18.168669 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:18.169513 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:18.169513 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:18.169513 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:18.169513 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:18.169513 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:56:18.169513 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:56:18.169513 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:56:18.169513 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:56:18.169513 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:56:18.169513 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:56:18.169513 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:56:18.169513 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:18.169513 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 311/642] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 13:56:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:18.170220 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f6d207-7a19-491a-99de-84a5b9457c6f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:18.249496 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a1da2b-37ed-48e4-8c27-84e200a83d3c None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:18.263487 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:18.272292 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f6d207-7a19-491a-99de-84a5b9457c6f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:18.275439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:18.287745 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a67cc-05d2-412d-97ab-c8c02c5cf979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:18.288399 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a67cc-05d2-412d-97ab-c8c02c5cf979 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:18.289220 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a67cc-05d2-412d-97ab-c8c02c5cf979 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:18.289490 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a67cc-05d2-412d-97ab-c8c02c5cf979 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:18.299885 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8a67cc-05d2-412d-97ab-c8c02c5cf979 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:56:18.305291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-ca8a67cc-05d2-412d-97ab-c8c02c5cf979 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:56:18.305291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:18.305291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:18.305291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:18.305291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:18.305291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:18.305291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:18.305291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:18.305291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:18.305291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:18.305291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:18.305291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:18.305291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:56:18.305291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:56:18.305291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:56:18.305291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:56:18.306291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:56:18.306291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:56:18.306291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:56:18.306291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:56:18.306291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:18.306291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:18.306291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:18.306291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:18.306291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:18.306291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:18.306291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:18.306291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:18.306291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:18.306291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:18.306291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:18.306291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:18.307885 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:18.307885 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:18.307885 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:18.307885 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:18.307885 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:56:18.307885 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:56:18.307885 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:56:18.307885 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:56:18.307885 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:56:18.307885 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:56:18.307885 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:56:18.307885 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:18.307885 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 332/643] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 13:56:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:18.307885 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a67cc-05d2-412d-97ab-c8c02c5cf979 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:18.315093 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a9f78f-449a-49b3-99e1-5bbbc848ab72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:18.315728 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9f78f-449a-49b3-99e1-5bbbc848ab72 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:18.316013 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9f78f-449a-49b3-99e1-5bbbc848ab72 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:18.316291 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9f78f-449a-49b3-99e1-5bbbc848ab72 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:18.317047 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a9f78f-449a-49b3-99e1-5bbbc848ab72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:18.317359 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a9f78f-449a-49b3-99e1-5bbbc848ab72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:18.318745 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a9f78f-449a-49b3-99e1-5bbbc848ab72 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:18.331554 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 312/644] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:18.332396 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a9f78f-449a-49b3-99e1-5bbbc848ab72 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:18.408770 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a67cc-05d2-412d-97ab-c8c02c5cf979 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:18.431385 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a9f78f-449a-49b3-99e1-5bbbc848ab72 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:18.442533 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0852a41-2ccb-4cd1-8f7c-cdcfe04ee596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:18.443206 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0852a41-2ccb-4cd1-8f7c-cdcfe04ee596 None admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:18.443483 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0852a41-2ccb-4cd1-8f7c-cdcfe04ee596 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:18.443755 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0852a41-2ccb-4cd1-8f7c-cdcfe04ee596 None admin] PATH_INFO: `/v3/projects/72ecec9fad964b1fbeaefd0519597229/users/2fe2a6d7bd10435ba2c8e552d808b7a0/roles/d3d8169e71454005a8cf0d7544ad05dc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:18.457864 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0852a41-2ccb-4cd1-8f7c-cdcfe04ee596 None admin] RBAC: Authorizing `identity:create_grant(project_id=72ecec9fad964b1fbeaefd0519597229, user_id=2fe2a6d7bd10435ba2c8e552d808b7a0, role_id=d3d8169e71454005a8cf0d7544ad05dc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:18.459300 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0852a41-2ccb-4cd1-8f7c-cdcfe04ee596 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:18.465857 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0338d5d5-7b43-46d8-b1b9-ad5f466cdd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:18.466514 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338d5d5-7b43-46d8-b1b9-ad5f466cdd0c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:18.467006 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338d5d5-7b43-46d8-b1b9-ad5f466cdd0c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:18.467282 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338d5d5-7b43-46d8-b1b9-ad5f466cdd0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:18.472077 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 333/645] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 13:56:18 2022] PUT /identity/v3/projects/72ecec9fad964b1fbeaefd0519597229/users/2fe2a6d7bd10435ba2c8e552d808b7a0/roles/d3d8169e71454005a8cf0d7544ad05dc => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:56:18.472949 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0852a41-2ccb-4cd1-8f7c-cdcfe04ee596 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:18.489619 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:18.511142 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:18.551036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0338d5d5-7b43-46d8-b1b9-ad5f466cdd0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:18.552439 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0338d5d5-7b43-46d8-b1b9-ad5f466cdd0c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:18.604520 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0852a41-2ccb-4cd1-8f7c-cdcfe04ee596 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:18.653588 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c39b6e-be23-44b9-b58d-26edcda09ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:18.654587 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c39b6e-be23-44b9-b58d-26edcda09ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:18.654882 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c39b6e-be23-44b9-b58d-26edcda09ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:18.655313 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c39b6e-be23-44b9-b58d-26edcda09ca0 None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:18.656056 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c39b6e-be23-44b9-b58d-26edcda09ca0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:18.656389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c39b6e-be23-44b9-b58d-26edcda09ca0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:18.657875 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c39b6e-be23-44b9-b58d-26edcda09ca0 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:18.667127 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 334/646] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 13:56:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:18.671846 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c39b6e-be23-44b9-b58d-26edcda09ca0 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:18.682920 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 313/647] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:56:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:56:18.684343 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0338d5d5-7b43-46d8-b1b9-ad5f466cdd0c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:18.736122 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:18.766663 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:18.774548 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c39b6e-be23-44b9-b58d-26edcda09ca0 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:18.794663 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0338d5d5-7b43-46d8-b1b9-ad5f466cdd0c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:18.812047 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99458e48-56cf-4db2-811b-a096784f94f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:18.812798 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99458e48-56cf-4db2-811b-a096784f94f6 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:18.813179 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99458e48-56cf-4db2-811b-a096784f94f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:18.813554 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99458e48-56cf-4db2-811b-a096784f94f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:18.814386 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99458e48-56cf-4db2-811b-a096784f94f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:18.814790 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99458e48-56cf-4db2-811b-a096784f94f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:18.816270 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99458e48-56cf-4db2-811b-a096784f94f6 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:18.825299 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 335/648] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 13:56:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:18.826199 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99458e48-56cf-4db2-811b-a096784f94f6 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:18.841034 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d4c7e-39b9-494d-8306-a887e5b7a023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:18.841685 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d4c7e-39b9-494d-8306-a887e5b7a023 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:18.841957 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d4c7e-39b9-494d-8306-a887e5b7a023 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:18.842239 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d4c7e-39b9-494d-8306-a887e5b7a023 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:18.849521 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0d4c7e-39b9-494d-8306-a887e5b7a023 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:18.850951 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0d4c7e-39b9-494d-8306-a887e5b7a023 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:18.863237 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-aa0d4c7e-39b9-494d-8306-a887e5b7a023 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:18.863237 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:18.863237 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:18.863237 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:18.863237 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:18.863237 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:18.863237 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:18.863237 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:18.863237 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:18.863237 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:18.863237 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:18.863237 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:18.863237 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:56:18.863237 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:56:18.863237 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:56:18.864103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:56:18.864103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:18.864103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:18.864103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:18.864103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:18.864103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:18.864103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:18.864103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:18.864103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:18.864103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:18.864103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:18.864103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:18.864103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:18.864103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:18.864103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:18.864103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:18.864103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:18.865029 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:56:18.865029 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:56:18.865029 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:56:18.865029 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:56:18.865029 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:56:18.865029 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:56:18.865029 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:18.865029 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:18.865029 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 314/649] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:56:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:18.865532 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d4c7e-39b9-494d-8306-a887e5b7a023 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:18.939037 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99458e48-56cf-4db2-811b-a096784f94f6 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:18.972965 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d4c7e-39b9-494d-8306-a887e5b7a023 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:18.973182 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57367cfe-b0fc-48ce-b2fa-e6a165479e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:18.973741 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57367cfe-b0fc-48ce-b2fa-e6a165479e62 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:18.974027 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57367cfe-b0fc-48ce-b2fa-e6a165479e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:18.974306 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57367cfe-b0fc-48ce-b2fa-e6a165479e62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:18.985582 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57367cfe-b0fc-48ce-b2fa-e6a165479e62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:18.986717 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:18.987033 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57367cfe-b0fc-48ce-b2fa-e6a165479e62 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:19.001310 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-57367cfe-b0fc-48ce-b2fa-e6a165479e62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:19.001310 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:19.001310 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:19.001310 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:19.001310 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:19.001310 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:19.001310 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:19.001310 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:19.001310 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:19.001310 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:19.001310 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:19.001310 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:19.001310 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:19.001310 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:19.001310 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:19.002223 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:19.002223 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:19.002223 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:19.002223 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:19.002223 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:19.002223 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:19.002223 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:19.002223 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:19.002223 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:19.002223 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:19.002223 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:19.002223 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:19.002223 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:19.002223 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:19.002223 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:19.002223 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:19.002223 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:19.003083 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:19.003083 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:19.003083 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:19.003083 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:19.003083 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 336/650] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:19.003370 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57367cfe-b0fc-48ce-b2fa-e6a165479e62 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:19.018172 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:19.021105 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af3066-8c71-47e7-ad4f-41d43779871d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:19.021849 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af3066-8c71-47e7-ad4f-41d43779871d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:19.022129 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af3066-8c71-47e7-ad4f-41d43779871d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:19.022400 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af3066-8c71-47e7-ad4f-41d43779871d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:19.035588 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af3066-8c71-47e7-ad4f-41d43779871d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:19.037086 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af3066-8c71-47e7-ad4f-41d43779871d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:19.053281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-33af3066-8c71-47e7-ad4f-41d43779871d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:19.053281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:19.053281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:19.053281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:19.053281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:19.053281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:19.053281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:19.053281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:19.053281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:19.053281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:19.053281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:19.053281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:19.053281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:19.053281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:19.053281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:19.054384 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:19.054384 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:19.054384 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:19.054384 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:19.054384 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:19.054384 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:19.054384 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:19.054384 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:19.054384 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:19.054384 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:19.054384 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:19.054384 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:19.054384 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:19.054384 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:19.054384 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:19.054384 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:19.054384 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:19.055279 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:19.055279 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:19.055279 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:19.055279 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:19.055279 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 315/651] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:19.055550 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af3066-8c71-47e7-ad4f-41d43779871d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:19.119390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57367cfe-b0fc-48ce-b2fa-e6a165479e62 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:19.155024 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998433b-be11-4538-9d9c-2854bcee43ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:19.155677 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998433b-be11-4538-9d9c-2854bcee43ac None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:19.155963 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998433b-be11-4538-9d9c-2854bcee43ac None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:19.156240 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998433b-be11-4538-9d9c-2854bcee43ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:19.162202 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af3066-8c71-47e7-ad4f-41d43779871d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:19.167466 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b998433b-be11-4538-9d9c-2854bcee43ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:19.168959 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b998433b-be11-4538-9d9c-2854bcee43ac None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:19.183482 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-b998433b-be11-4538-9d9c-2854bcee43ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:19.183482 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:19.183482 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:19.183482 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:19.183482 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:19.183482 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:19.183482 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:19.183482 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:19.183482 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:19.183482 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:19.183482 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:19.183482 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:19.183482 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:19.183482 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:19.183482 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:19.184389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:19.184389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:19.184389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:19.184389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:19.184389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:19.184389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:19.184389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:19.184389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:19.184389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:19.184389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:19.184389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:19.184389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:19.184389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:19.184389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:19.184389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:19.184389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:19.184389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:19.185405 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:19.185405 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:19.185405 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:19.185405 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:19.185405 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 337/652] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:19.185405 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998433b-be11-4538-9d9c-2854bcee43ac None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:19.202619 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1cc0b-c2b9-4604-b5c0-d8afb4959a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:19.203243 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1cc0b-c2b9-4604-b5c0-d8afb4959a4a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:19.203521 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1cc0b-c2b9-4604-b5c0-d8afb4959a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:19.203794 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1cc0b-c2b9-4604-b5c0-d8afb4959a4a None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:19.204579 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b1cc0b-c2b9-4604-b5c0-d8afb4959a4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:19.204887 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b1cc0b-c2b9-4604-b5c0-d8afb4959a4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:19.206317 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b1cc0b-c2b9-4604-b5c0-d8afb4959a4a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:19.219232 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 316/653] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 13:56:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:19.220034 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1cc0b-c2b9-4604-b5c0-d8afb4959a4a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:19.245222 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:19.255409 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:19.302126 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998433b-be11-4538-9d9c-2854bcee43ac None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:19.328933 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1cc0b-c2b9-4604-b5c0-d8afb4959a4a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:19.341182 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baab4f4-a136-4bdf-86b5-dc199aaad3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:19.341806 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baab4f4-a136-4bdf-86b5-dc199aaad3f4 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:19.342093 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baab4f4-a136-4bdf-86b5-dc199aaad3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:19.342511 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baab4f4-a136-4bdf-86b5-dc199aaad3f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:19.343564 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baab4f4-a136-4bdf-86b5-dc199aaad3f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:19.343877 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baab4f4-a136-4bdf-86b5-dc199aaad3f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:19.345441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baab4f4-a136-4bdf-86b5-dc199aaad3f4 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:19.358135 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 338/654] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:19.359046 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baab4f4-a136-4bdf-86b5-dc199aaad3f4 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:19.370220 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dab30c-9ed1-4b1f-916f-8df2ec9f29b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:19.371093 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dab30c-9ed1-4b1f-916f-8df2ec9f29b1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:19.371612 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dab30c-9ed1-4b1f-916f-8df2ec9f29b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:19.372134 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dab30c-9ed1-4b1f-916f-8df2ec9f29b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:19.373127 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dab30c-9ed1-4b1f-916f-8df2ec9f29b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:19.373489 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dab30c-9ed1-4b1f-916f-8df2ec9f29b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:19.374820 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dab30c-9ed1-4b1f-916f-8df2ec9f29b1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:19.395702 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 317/655] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:19.395702 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dab30c-9ed1-4b1f-916f-8df2ec9f29b1 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:19.470068 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baab4f4-a136-4bdf-86b5-dc199aaad3f4 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:19.481583 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:19.493426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:19.504247 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dab30c-9ed1-4b1f-916f-8df2ec9f29b1 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:19.510076 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bdbe3b-0b5f-43a6-a885-22183b048358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:19.510701 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bdbe3b-0b5f-43a6-a885-22183b048358 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:19.510985 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bdbe3b-0b5f-43a6-a885-22183b048358 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:19.511263 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bdbe3b-0b5f-43a6-a885-22183b048358 None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:19.511959 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bdbe3b-0b5f-43a6-a885-22183b048358 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:19.512266 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bdbe3b-0b5f-43a6-a885-22183b048358 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:19.513680 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bdbe3b-0b5f-43a6-a885-22183b048358 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:19.525928 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 339/656] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:19.526734 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bdbe3b-0b5f-43a6-a885-22183b048358 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:19.545167 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eaa647-c63d-437d-b6f0-3e175386e588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:19.546252 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eaa647-c63d-437d-b6f0-3e175386e588 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:19.546622 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eaa647-c63d-437d-b6f0-3e175386e588 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:19.546967 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eaa647-c63d-437d-b6f0-3e175386e588 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:19.547832 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eaa647-c63d-437d-b6f0-3e175386e588 None admin] RBAC: Adding query filter params (role.id=d3d8169e71454005a8cf0d7544ad05dc, scope.project.id=72ecec9fad964b1fbeaefd0519597229, user.id=a246654c783b4cc68d212e91bb05d8c6) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:19.548213 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eaa647-c63d-437d-b6f0-3e175386e588 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:19.549748 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eaa647-c63d-437d-b6f0-3e175386e588 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:19.564902 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 318/657] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:56:19 2022] GET /identity/v3/role_assignments?user.id=a246654c783b4cc68d212e91bb05d8c6&scope.project.id=72ecec9fad964b1fbeaefd0519597229&role.id=d3d8169e71454005a8cf0d7544ad05dc => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:19.567404 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eaa647-c63d-437d-b6f0-3e175386e588 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:19.646315 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bdbe3b-0b5f-43a6-a885-22183b048358 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:19.688116 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eaa647-c63d-437d-b6f0-3e175386e588 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:19.703057 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4f830-176a-4cbf-9edd-04e5dab41034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:19.703826 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4f830-176a-4cbf-9edd-04e5dab41034 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:19.704201 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4f830-176a-4cbf-9edd-04e5dab41034 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:19.704572 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4f830-176a-4cbf-9edd-04e5dab41034 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:19.713845 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a4f830-176a-4cbf-9edd-04e5dab41034 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:19.715441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a4f830-176a-4cbf-9edd-04e5dab41034 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:19.727671 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-69a4f830-176a-4cbf-9edd-04e5dab41034 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 13:56:19.727671 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:19.727671 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:19.727671 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:19.727671 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:19.727671 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:19.727671 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:19.727671 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:19.727671 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:19.727671 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:19.727671 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:19.727671 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:19.727671 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:56:19.727671 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:56:19.727671 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:56:19.727671 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:56:19.728815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:19.728815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:19.728815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:56:19.728815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:19.728815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:56:19.728815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:19.728815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:19.728815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:19.728815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:19.728815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:19.728815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:19.728815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:19.728815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:19.728815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:19.728815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:19.728815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:19.728815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:19.728815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:19.729814 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:19.729814 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:19.729814 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:56:19.729814 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:56:19.729814 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:56:19.729814 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:56:19.729814 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:56:19.729814 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:56:19.729814 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 13:56:19.729814 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:19.729814 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 340/658] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:56:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:19.730385 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4f830-176a-4cbf-9edd-04e5dab41034 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:19.732019 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf3da8c-62a0-4f0b-82be-91bdab983f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:19.732866 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3da8c-62a0-4f0b-82be-91bdab983f6c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:19.733288 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3da8c-62a0-4f0b-82be-91bdab983f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:19.733702 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3da8c-62a0-4f0b-82be-91bdab983f6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:19.742299 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf3da8c-62a0-4f0b-82be-91bdab983f6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:19.743962 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf3da8c-62a0-4f0b-82be-91bdab983f6c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:19.755840 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-5bf3da8c-62a0-4f0b-82be-91bdab983f6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:19.755840 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:19.755840 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:19.755840 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:19.755840 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:19.755840 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:19.755840 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:19.755840 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:19.755840 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:19.755840 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:19.755840 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:19.755840 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:19.755840 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:56:19.755840 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:56:19.755840 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:56:19.757107 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:56:19.757107 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:19.757107 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:19.757107 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:19.757107 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:19.757107 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:19.757107 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:19.757107 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:19.757107 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:19.757107 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:19.757107 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:19.757107 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:19.757107 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:19.757107 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:19.757107 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:19.757107 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:19.757107 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:19.758551 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:56:19.758551 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:56:19.758551 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:56:19.758551 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:56:19.758551 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:56:19.758551 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:56:19.758551 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:19.758551 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:19.758551 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 319/659] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:56:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:19.759627 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf3da8c-62a0-4f0b-82be-91bdab983f6c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:19.768257 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:19.787766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:19.872320 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4f830-176a-4cbf-9edd-04e5dab41034 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:19.890861 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf3da8c-62a0-4f0b-82be-91bdab983f6c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:19.918127 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ce5ec-33ad-4aa0-8933-293913874d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:19.919177 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ce5ec-33ad-4aa0-8933-293913874d7e None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:19.919728 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ce5ec-33ad-4aa0-8933-293913874d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:19.920010 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ce5ec-33ad-4aa0-8933-293913874d7e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:19.928295 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83425c4-1094-47ab-8439-a1926cc8dabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:19.928952 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83425c4-1094-47ab-8439-a1926cc8dabc None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:19.929803 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83425c4-1094-47ab-8439-a1926cc8dabc None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:19.929929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3ce5ec-33ad-4aa0-8933-293913874d7e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:19.930100 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83425c4-1094-47ab-8439-a1926cc8dabc None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:19.930894 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83425c4-1094-47ab-8439-a1926cc8dabc None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:19.931206 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83425c4-1094-47ab-8439-a1926cc8dabc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:19.931310 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3ce5ec-33ad-4aa0-8933-293913874d7e None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:19.932744 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83425c4-1094-47ab-8439-a1926cc8dabc None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:19.945807 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-6a3ce5ec-33ad-4aa0-8933-293913874d7e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 13:56:19.945807 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:19.945807 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:19.945807 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:19.945807 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:19.945807 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:19.945807 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:19.945807 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:19.945807 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:19.945807 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:19.945807 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:19.945807 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:19.945807 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:56:19.945807 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:56:19.945807 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:56:19.945807 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:56:19.946766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:19.946766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:19.946766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:56:19.946766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:19.946766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:56:19.946766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:19.946766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:19.946766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:19.946766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:19.946766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:19.946766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:19.946766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:19.946766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:19.946766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:19.946766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:19.946766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:19.946766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:19.946766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:19.947672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:19.947672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:19.947672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:56:19.947672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:56:19.947672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:56:19.947672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:56:19.947672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:56:19.947672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:56:19.947672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 13:56:19.947672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:19.947672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 341/660] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 13:56:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:19.948975 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ce5ec-33ad-4aa0-8933-293913874d7e None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:19.952716 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 320/661] 10.222.0.48 () {56 vars in 1137 bytes} [Mon Aug 1 13:56:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:19.953727 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83425c4-1094-47ab-8439-a1926cc8dabc None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:20.034518 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:20.071368 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:20.075006 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ce5ec-33ad-4aa0-8933-293913874d7e None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:20.090409 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83425c4-1094-47ab-8439-a1926cc8dabc None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:20.121041 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d3b944-679e-4f08-aea8-2b10fd5b8460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:20.122872 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3b944-679e-4f08-aea8-2b10fd5b8460 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:20.123325 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3b944-679e-4f08-aea8-2b10fd5b8460 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:20.123977 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3b944-679e-4f08-aea8-2b10fd5b8460 None admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:20.125021 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d3b944-679e-4f08-aea8-2b10fd5b8460 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:20.125475 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d3b944-679e-4f08-aea8-2b10fd5b8460 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:20.127260 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d3b944-679e-4f08-aea8-2b10fd5b8460 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:20.127985 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3d29a1-030d-4076-9a4d-206699283716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:20.128763 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d29a1-030d-4076-9a4d-206699283716 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:20.129166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d29a1-030d-4076-9a4d-206699283716 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:20.129651 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d29a1-030d-4076-9a4d-206699283716 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:20.130492 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3d29a1-030d-4076-9a4d-206699283716 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:20.131439 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3d29a1-030d-4076-9a4d-206699283716 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:20.133494 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3d29a1-030d-4076-9a4d-206699283716 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:20.145242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 342/662] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 13:56:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:20.146420 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d3b944-679e-4f08-aea8-2b10fd5b8460 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:20.153906 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 321/663] 10.222.0.48 () {56 vars in 1139 bytes} [Mon Aug 1 13:56:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:20.155120 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3d29a1-030d-4076-9a4d-206699283716 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:20.265928 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d3b944-679e-4f08-aea8-2b10fd5b8460 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:20.276268 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3d29a1-030d-4076-9a4d-206699283716 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:20.300758 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:20.319077 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6dd0fc-9f81-499e-bf9c-88fe2ffa1397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:20.319077 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6dd0fc-9f81-499e-bf9c-88fe2ffa1397 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:20.319418 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c83c3b-d5cb-4cea-9ff6-c0c582119d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:20.319530 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6dd0fc-9f81-499e-bf9c-88fe2ffa1397 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:20.319530 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6dd0fc-9f81-499e-bf9c-88fe2ffa1397 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:20.319684 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c83c3b-d5cb-4cea-9ff6-c0c582119d50 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:20.319684 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c83c3b-d5cb-4cea-9ff6-c0c582119d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:20.319684 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c83c3b-d5cb-4cea-9ff6-c0c582119d50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:20.320727 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c83c3b-d5cb-4cea-9ff6-c0c582119d50 None admin] RBAC: Adding query filter params (role.id=d3d8169e71454005a8cf0d7544ad05dc, scope.project.id=72ecec9fad964b1fbeaefd0519597229, user.id=b185b052f9df4d3f9847d8f9b4cfda5f) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:20.321143 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c83c3b-d5cb-4cea-9ff6-c0c582119d50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:20.322727 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c83c3b-d5cb-4cea-9ff6-c0c582119d50 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:20.336505 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 322/664] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:56:20 2022] GET /identity/v3/role_assignments?user.id=b185b052f9df4d3f9847d8f9b4cfda5f&scope.project.id=72ecec9fad964b1fbeaefd0519597229&role.id=d3d8169e71454005a8cf0d7544ad05dc => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:20.336505 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c83c3b-d5cb-4cea-9ff6-c0c582119d50 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:20.408914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:20.457058 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6dd0fc-9f81-499e-bf9c-88fe2ffa1397 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:20.459189 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6dd0fc-9f81-499e-bf9c-88fe2ffa1397 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:20.511963 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c83c3b-d5cb-4cea-9ff6-c0c582119d50 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:20.556159 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a7496-7ce4-48c7-a25a-d49efe28c6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:20.556745 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a7496-7ce4-48c7-a25a-d49efe28c6cc None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:20.557030 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a7496-7ce4-48c7-a25a-d49efe28c6cc None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:20.557301 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a7496-7ce4-48c7-a25a-d49efe28c6cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:20.620044 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:20.627535 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 343/665] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:56:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:56:20.629382 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528bc89-c89c-4dde-b01a-ebf3299d1c77 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:20.629820 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528bc89-c89c-4dde-b01a-ebf3299d1c77 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:20.630246 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528bc89-c89c-4dde-b01a-ebf3299d1c77 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:20.632170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 344/666] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:20.639024 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f3843-fada-4022-b7db-5ace546b0d02 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:20.639508 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f3843-fada-4022-b7db-5ace546b0d02 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:20.639933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f3843-fada-4022-b7db-5ace546b0d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:20.644318 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851a7496-7ce4-48c7-a25a-d49efe28c6cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:20.645789 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851a7496-7ce4-48c7-a25a-d49efe28c6cc None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:20.663029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-dc4f3843-fada-4022-b7db-5ace546b0d02 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:20.665718 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-dc4f3843-fada-4022-b7db-5ace546b0d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:20.739603 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:20.757011 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 345/667] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:56:20.758636 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f3843-fada-4022-b7db-5ace546b0d02 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:20.789788 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 323/668] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:56:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:56:20.790782 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a7496-7ce4-48c7-a25a-d49efe28c6cc None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:20.912019 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f3843-fada-4022-b7db-5ace546b0d02 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:20.932513 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:20.932513 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a7496-7ce4-48c7-a25a-d49efe28c6cc None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:20.954994 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988aaaf6-0de2-4ea4-8d85-027f46077a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:20.956095 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988aaaf6-0de2-4ea4-8d85-027f46077a42 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:20.956497 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988aaaf6-0de2-4ea4-8d85-027f46077a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:20.956881 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988aaaf6-0de2-4ea4-8d85-027f46077a42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:20.971320 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988aaaf6-0de2-4ea4-8d85-027f46077a42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:20.972953 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988aaaf6-0de2-4ea4-8d85-027f46077a42 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:20.975216 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82149628-a68c-4dde-b2ab-c278f1725d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:20.976951 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82149628-a68c-4dde-b2ab-c278f1725d45 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:20.977310 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82149628-a68c-4dde-b2ab-c278f1725d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:20.977686 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82149628-a68c-4dde-b2ab-c278f1725d45 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:20.989908 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82149628-a68c-4dde-b2ab-c278f1725d45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:56:20.990781 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-988aaaf6-0de2-4ea4-8d85-027f46077a42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:20.990781 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:20.990781 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:20.990781 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:20.990781 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:20.990781 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:20.990781 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:20.990781 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:20.990781 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:20.990781 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:20.990781 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:20.990781 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:20.990781 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:20.990781 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:20.990781 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:20.992332 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:20.992332 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:20.992332 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:20.992332 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:20.992332 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:20.992332 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:20.992332 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:20.992332 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:20.992332 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:20.992332 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:20.992332 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:20.992332 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:20.992332 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:20.992332 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:20.992332 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:20.992332 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:20.992332 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:20.993431 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:20.993431 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:20.993431 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:20.993431 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:21.000504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 346/669] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:21.000504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988aaaf6-0de2-4ea4-8d85-027f46077a42 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:21.000772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-82149628-a68c-4dde-b2ab-c278f1725d45 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:56:21.000772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:21.000772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:21.000772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:21.000772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:21.000772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:21.000772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:21.000772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:21.000772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:21.000772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:21.000772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:21.000772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:21.000772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:56:21.000772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:56:21.000772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:56:21.000772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:56:21.001795 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:56:21.001795 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:56:21.001795 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:56:21.001795 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:56:21.001795 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:21.001795 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:21.001795 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:21.001795 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:21.001795 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:21.001795 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:21.001795 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:21.001795 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:21.001795 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:21.001795 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:21.001795 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:21.001795 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:21.003302 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:21.003302 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:21.003302 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:21.003302 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:21.003302 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:56:21.003302 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:56:21.003302 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:56:21.003302 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:56:21.003302 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:56:21.003302 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:56:21.003302 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:56:21.003302 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:21.003302 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 324/670] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 13:56:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:21.003302 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82149628-a68c-4dde-b2ab-c278f1725d45 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:21.057152 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:21.119735 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988aaaf6-0de2-4ea4-8d85-027f46077a42 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:21.121747 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82149628-a68c-4dde-b2ab-c278f1725d45 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:21.157088 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbfff5c-6b20-4cb5-9bdf-457a804381d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:21.157731 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbfff5c-6b20-4cb5-9bdf-457a804381d6 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:21.158112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbfff5c-6b20-4cb5-9bdf-457a804381d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:21.158393 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbfff5c-6b20-4cb5-9bdf-457a804381d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:21.158576 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55fd457-8a33-43c0-8885-7a7e02a32546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:21.159243 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55fd457-8a33-43c0-8885-7a7e02a32546 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:21.159529 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55fd457-8a33-43c0-8885-7a7e02a32546 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:21.159806 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55fd457-8a33-43c0-8885-7a7e02a32546 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:21.160535 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55fd457-8a33-43c0-8885-7a7e02a32546 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:21.160851 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55fd457-8a33-43c0-8885-7a7e02a32546 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:21.162226 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55fd457-8a33-43c0-8885-7a7e02a32546 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:21.171292 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbfff5c-6b20-4cb5-9bdf-457a804381d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:21.172783 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbfff5c-6b20-4cb5-9bdf-457a804381d6 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:21.179364 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 325/671] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:21.180270 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55fd457-8a33-43c0-8885-7a7e02a32546 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:21.188888 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-adbfff5c-6b20-4cb5-9bdf-457a804381d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:21.188888 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:21.188888 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:21.188888 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:21.188888 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:21.188888 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:21.188888 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:21.188888 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:21.188888 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:21.188888 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:21.188888 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:21.188888 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:21.188888 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:21.188888 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:21.188888 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:21.190910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:21.190910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:21.190910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:21.190910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:21.190910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:21.190910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:21.190910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:21.190910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:21.190910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:21.190910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:21.190910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:21.190910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:21.190910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:21.190910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:21.190910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:21.190910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:21.190910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:21.192883 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:21.192883 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:21.192883 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:21.192883 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:21.197157 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:21.197321 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 347/672] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:21.197321 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbfff5c-6b20-4cb5-9bdf-457a804381d6 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:21.291351 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55fd457-8a33-43c0-8885-7a7e02a32546 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:21.292610 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:21.300741 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbfff5c-6b20-4cb5-9bdf-457a804381d6 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:21.329453 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c028f-d7e2-4814-885b-2ef4a9b51543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:21.330079 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c028f-d7e2-4814-885b-2ef4a9b51543 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:21.330362 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c028f-d7e2-4814-885b-2ef4a9b51543 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:21.330649 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c028f-d7e2-4814-885b-2ef4a9b51543 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:21.331550 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c028f-d7e2-4814-885b-2ef4a9b51543 None admin] RBAC: Adding query filter params (name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:21.331863 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c028f-d7e2-4814-885b-2ef4a9b51543 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:21.333255 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c028f-d7e2-4814-885b-2ef4a9b51543 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:21.337352 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc09e4fb-d489-4eed-b13e-73fe0fd21961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:21.338003 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09e4fb-d489-4eed-b13e-73fe0fd21961 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:21.338288 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09e4fb-d489-4eed-b13e-73fe0fd21961 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:21.338626 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09e4fb-d489-4eed-b13e-73fe0fd21961 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:21.345922 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc09e4fb-d489-4eed-b13e-73fe0fd21961 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:21.347486 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc09e4fb-d489-4eed-b13e-73fe0fd21961 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:21.347792 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 326/673] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 13:56:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:21.348884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c028f-d7e2-4814-885b-2ef4a9b51543 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:21.359722 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-bc09e4fb-d489-4eed-b13e-73fe0fd21961 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:21.359722 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:21.359722 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:21.359722 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:21.359722 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:21.359722 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:21.359722 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:21.359722 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:21.359722 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:21.359722 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:21.359722 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:21.359722 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:21.359722 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:56:21.359722 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:56:21.359722 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:56:21.361772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:56:21.361772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:21.361772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:21.361772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:21.361772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:21.361772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:21.361772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:21.361772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:21.361772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:21.361772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:21.361772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:21.361772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:21.361772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:21.361772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:21.361772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:21.361772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:21.361772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:21.363819 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:56:21.363819 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:56:21.363819 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:56:21.363819 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:56:21.363819 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:56:21.363819 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:56:21.363819 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:21.363819 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:21.363819 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 348/674] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:56:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:21.363819 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc09e4fb-d489-4eed-b13e-73fe0fd21961 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:21.425476 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:21.465224 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c028f-d7e2-4814-885b-2ef4a9b51543 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:21.468539 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc09e4fb-d489-4eed-b13e-73fe0fd21961 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:21.502389 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3175f12b-56d5-4779-bce0-28c7dfc61b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:21.503291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182eb587-2007-4ee4-ba37-9d9afcc87cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:21.503466 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175f12b-56d5-4779-bce0-28c7dfc61b2c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:21.503885 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175f12b-56d5-4779-bce0-28c7dfc61b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:21.503999 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182eb587-2007-4ee4-ba37-9d9afcc87cfc None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:21.504210 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182eb587-2007-4ee4-ba37-9d9afcc87cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:21.504510 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182eb587-2007-4ee4-ba37-9d9afcc87cfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:21.504735 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175f12b-56d5-4779-bce0-28c7dfc61b2c None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:21.505767 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3175f12b-56d5-4779-bce0-28c7dfc61b2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:21.506568 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3175f12b-56d5-4779-bce0-28c7dfc61b2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:21.508789 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3175f12b-56d5-4779-bce0-28c7dfc61b2c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:21.516567 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182eb587-2007-4ee4-ba37-9d9afcc87cfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:21.517922 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182eb587-2007-4ee4-ba37-9d9afcc87cfc None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:21.527145 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 327/675] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:21.528326 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3175f12b-56d5-4779-bce0-28c7dfc61b2c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:21.534242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-182eb587-2007-4ee4-ba37-9d9afcc87cfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:21.534242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:21.534242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:21.534242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:21.534242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:21.534242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:21.534242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:21.534242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:21.534242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:21.534242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:21.534242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:21.534242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:21.534242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:21.534242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:21.534242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:21.535302 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:21.535302 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:21.535302 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:21.535302 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:21.535302 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:21.535302 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:21.535302 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:21.535302 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:21.535302 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:21.535302 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:21.535302 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:21.535302 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:21.535302 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:21.535302 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:21.535302 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:21.535302 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:21.535302 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:21.536320 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:21.536320 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:21.536320 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:21.536320 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:21.540453 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 349/676] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:21.541336 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182eb587-2007-4ee4-ba37-9d9afcc87cfc None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:21.551760 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:21.645496 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3175f12b-56d5-4779-bce0-28c7dfc61b2c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:21.652776 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182eb587-2007-4ee4-ba37-9d9afcc87cfc None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:21.676058 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:21.686412 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9a3940-8d58-4143-a487-cf7a02ebf0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:21.687494 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a3940-8d58-4143-a487-cf7a02ebf0d5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:21.688013 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a3940-8d58-4143-a487-cf7a02ebf0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:21.688536 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a3940-8d58-4143-a487-cf7a02ebf0d5 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:21.689406 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921511bb-7ae8-44a5-a73d-8a31bfa7b76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:21.689809 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9a3940-8d58-4143-a487-cf7a02ebf0d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:21.690054 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921511bb-7ae8-44a5-a73d-8a31bfa7b76e None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:21.690342 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921511bb-7ae8-44a5-a73d-8a31bfa7b76e None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:21.690624 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921511bb-7ae8-44a5-a73d-8a31bfa7b76e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:21.690947 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9a3940-8d58-4143-a487-cf7a02ebf0d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:21.692866 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9a3940-8d58-4143-a487-cf7a02ebf0d5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:21.697806 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921511bb-7ae8-44a5-a73d-8a31bfa7b76e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:21.699298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921511bb-7ae8-44a5-a73d-8a31bfa7b76e None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:21.711601 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 328/677] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 13:56:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:21.711914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-921511bb-7ae8-44a5-a73d-8a31bfa7b76e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:21.711914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:21.711914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:21.711914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:21.711914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:21.711914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:21.711914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:21.711914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:21.711914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:21.711914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:21.711914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:21.711914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:21.711914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:56:21.711914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:56:21.711914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:56:21.712803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:56:21.712803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:21.712803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:21.712803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:21.712803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:21.712803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:21.712803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:21.712803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:21.712803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:21.712803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:21.712803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:21.712803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:21.712803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:21.712803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:21.712803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:21.712803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:21.712803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:21.713692 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:56:21.713692 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:56:21.713692 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:56:21.713692 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:56:21.713692 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:56:21.713692 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:56:21.713692 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:21.713692 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:21.713692 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 350/678] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:56:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:21.714182 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921511bb-7ae8-44a5-a73d-8a31bfa7b76e None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:21.715558 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9a3940-8d58-4143-a487-cf7a02ebf0d5 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:21.783664 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:21.824462 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9a3940-8d58-4143-a487-cf7a02ebf0d5 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:21.826058 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921511bb-7ae8-44a5-a73d-8a31bfa7b76e None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:21.859775 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c3d5ab-e1c0-4e9d-bdbe-88ddbe9c73bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:21.860399 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c3d5ab-e1c0-4e9d-bdbe-88ddbe9c73bf None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:21.860713 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c3d5ab-e1c0-4e9d-bdbe-88ddbe9c73bf None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:21.860991 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c3d5ab-e1c0-4e9d-bdbe-88ddbe9c73bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:21.861783 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a19559-cf41-47f1-b07c-49ae19b8bf24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:21.861897 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c3d5ab-e1c0-4e9d-bdbe-88ddbe9c73bf None admin] RBAC: Adding query filter params (role.id=d3d8169e71454005a8cf0d7544ad05dc, scope.project.id=72ecec9fad964b1fbeaefd0519597229, user.id=39a66f7362964389be039937a786bb0a) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:21.862095 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c3d5ab-e1c0-4e9d-bdbe-88ddbe9c73bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:21.862406 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a19559-cf41-47f1-b07c-49ae19b8bf24 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:21.862693 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a19559-cf41-47f1-b07c-49ae19b8bf24 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:21.862972 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a19559-cf41-47f1-b07c-49ae19b8bf24 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:21.863520 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c3d5ab-e1c0-4e9d-bdbe-88ddbe9c73bf None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:21.863680 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a19559-cf41-47f1-b07c-49ae19b8bf24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:21.863968 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a19559-cf41-47f1-b07c-49ae19b8bf24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:21.865467 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a19559-cf41-47f1-b07c-49ae19b8bf24 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:21.869448 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 351/679] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:56:21 2022] GET /identity/v3/role_assignments?user.id=39a66f7362964389be039937a786bb0a&scope.project.id=72ecec9fad964b1fbeaefd0519597229&role.id=d3d8169e71454005a8cf0d7544ad05dc => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:21.877578 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c3d5ab-e1c0-4e9d-bdbe-88ddbe9c73bf None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:21.880661 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 329/680] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:21.882715 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b6fe9-832d-4c2e-8273-b3a0535868bd None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:21.883731 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b6fe9-832d-4c2e-8273-b3a0535868bd None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:21.884016 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b6fe9-832d-4c2e-8273-b3a0535868bd None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:21.888663 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 330/681] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:21.890176 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b6fe9-832d-4c2e-8273-b3a0535868bd None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:21.957797 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:22.023768 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b6fe9-832d-4c2e-8273-b3a0535868bd None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:22.037325 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c3d5ab-e1c0-4e9d-bdbe-88ddbe9c73bf None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:22.063368 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa26bdf7-cf1d-4924-b604-b91d2b309f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:22.064375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26bdf7-cf1d-4924-b604-b91d2b309f79 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:22.064698 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26bdf7-cf1d-4924-b604-b91d2b309f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:22.065092 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26bdf7-cf1d-4924-b604-b91d2b309f79 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:22.072904 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:22.075197 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa26bdf7-cf1d-4924-b604-b91d2b309f79 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:22.077385 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa26bdf7-cf1d-4924-b604-b91d2b309f79 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:22.079021 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f0ee5-2c21-46a6-983f-2d2cfbebd642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:22.079897 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f0ee5-2c21-46a6-983f-2d2cfbebd642 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:22.080723 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f0ee5-2c21-46a6-983f-2d2cfbebd642 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:22.081730 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f0ee5-2c21-46a6-983f-2d2cfbebd642 None admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:22.081844 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027f0ee5-2c21-46a6-983f-2d2cfbebd642 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:22.082795 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027f0ee5-2c21-46a6-983f-2d2cfbebd642 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:22.083520 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027f0ee5-2c21-46a6-983f-2d2cfbebd642 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:22.092700 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-aa26bdf7-cf1d-4924-b604-b91d2b309f79 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 13:56:22.092700 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:22.092700 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:22.092700 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:22.092700 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:22.092700 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:22.092700 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:22.092700 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:22.092700 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:22.092700 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:22.092700 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:22.092700 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:22.092700 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:56:22.092700 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:56:22.092700 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:56:22.092700 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:56:22.093653 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:22.093653 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:22.093653 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:56:22.093653 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:22.093653 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:56:22.093653 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:22.093653 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:22.093653 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:22.093653 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:22.093653 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:22.093653 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:22.093653 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:22.093653 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:22.093653 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:22.093653 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:22.093653 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:22.093653 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:22.093653 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:22.094570 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:22.094570 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:22.094570 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:56:22.094570 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:56:22.094570 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:56:22.094570 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:56:22.094570 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:56:22.094570 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:56:22.094570 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 13:56:22.094570 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:22.094570 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 331/682] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 13:56:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:22.095151 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9234c8-a51b-430b-89f3-bd8f111d96aa None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:22.095217 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9234c8-a51b-430b-89f3-bd8f111d96aa None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:22.096272 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9234c8-a51b-430b-89f3-bd8f111d96aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:22.102915 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 352/683] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 13:56:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:22.104175 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f0ee5-2c21-46a6-983f-2d2cfbebd642 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:22.117697 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ac9234c8-a51b-430b-89f3-bd8f111d96aa None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:22.119201 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ac9234c8-a51b-430b-89f3-bd8f111d96aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:22.209214 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 332/684] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:56:22.216674 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9234c8-a51b-430b-89f3-bd8f111d96aa None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:22.243731 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f0ee5-2c21-46a6-983f-2d2cfbebd642 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:22.279481 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:22.283208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e46dc75-bb58-4f84-a13d-26a7ec9de5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:22.284822 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46dc75-bb58-4f84-a13d-26a7ec9de5f6 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:22.285180 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46dc75-bb58-4f84-a13d-26a7ec9de5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:22.285385 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46dc75-bb58-4f84-a13d-26a7ec9de5f6 None admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:22.286201 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e46dc75-bb58-4f84-a13d-26a7ec9de5f6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:22.286514 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e46dc75-bb58-4f84-a13d-26a7ec9de5f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:22.288267 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e46dc75-bb58-4f84-a13d-26a7ec9de5f6 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:22.308263 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 353/685] 10.222.0.48 () {56 vars in 1135 bytes} [Mon Aug 1 13:56:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:22.309271 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e46dc75-bb58-4f84-a13d-26a7ec9de5f6 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:22.359188 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9234c8-a51b-430b-89f3-bd8f111d96aa None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:22.368442 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:22.407855 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14ffbf4-ae6c-4d2f-b5b4-3e7dc66ea024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:22.409079 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ffbf4-ae6c-4d2f-b5b4-3e7dc66ea024 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:22.409548 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ffbf4-ae6c-4d2f-b5b4-3e7dc66ea024 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:22.410245 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ffbf4-ae6c-4d2f-b5b4-3e7dc66ea024 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:22.411690 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14ffbf4-ae6c-4d2f-b5b4-3e7dc66ea024 None admin] RBAC: Adding query filter params (role.id=d3d8169e71454005a8cf0d7544ad05dc, scope.project.id=72ecec9fad964b1fbeaefd0519597229, user.id=fb9f698e09d246bcbea855613764fbe1) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:22.412189 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14ffbf4-ae6c-4d2f-b5b4-3e7dc66ea024 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:22.414902 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14ffbf4-ae6c-4d2f-b5b4-3e7dc66ea024 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:22.423062 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 333/686] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:56:22 2022] GET /identity/v3/role_assignments?user.id=fb9f698e09d246bcbea855613764fbe1&scope.project.id=72ecec9fad964b1fbeaefd0519597229&role.id=d3d8169e71454005a8cf0d7544ad05dc => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:22.424286 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14ffbf4-ae6c-4d2f-b5b4-3e7dc66ea024 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:22.446109 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e46dc75-bb58-4f84-a13d-26a7ec9de5f6 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:22.503920 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4467fe40-a4a3-40b6-bf59-54b90e094182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:22.504580 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4467fe40-a4a3-40b6-bf59-54b90e094182 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:22.504868 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4467fe40-a4a3-40b6-bf59-54b90e094182 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:22.505190 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4467fe40-a4a3-40b6-bf59-54b90e094182 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:22.518353 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4467fe40-a4a3-40b6-bf59-54b90e094182 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:56:22.524394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-4467fe40-a4a3-40b6-bf59-54b90e094182 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:56:22.524394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:22.524394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:22.524394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:22.524394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:22.524394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:22.524394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:22.524394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:22.524394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:22.524394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:22.524394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:22.524394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:22.524394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:56:22.524394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:56:22.524394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:56:22.524394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:56:22.525504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:56:22.525504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:56:22.525504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:56:22.525504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:56:22.525504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:22.525504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:22.525504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:22.525504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:22.525504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:22.525504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:22.525504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:22.525504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:22.525504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:22.525504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:22.525504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:22.525504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:22.526342 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:22.526342 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:22.526342 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:22.526342 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:22.526342 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:56:22.526342 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:56:22.526342 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:56:22.526342 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:56:22.526342 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:56:22.526342 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:56:22.526342 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:56:22.526342 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:22.530796 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 354/687] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 13:56:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:22.532074 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a839b8e-265c-4319-95b9-09929041f5ee None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:22.532364 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a839b8e-265c-4319-95b9-09929041f5ee None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:22.532668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a839b8e-265c-4319-95b9-09929041f5ee None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:22.534453 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 355/688] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:22.535497 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a839b8e-265c-4319-95b9-09929041f5ee None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:22.549963 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14ffbf4-ae6c-4d2f-b5b4-3e7dc66ea024 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:22.552021 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:22.590134 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97184ab6-bb4b-4bea-aad2-d04651a42460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:22.591125 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97184ab6-bb4b-4bea-aad2-d04651a42460 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:22.591706 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97184ab6-bb4b-4bea-aad2-d04651a42460 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:22.592166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97184ab6-bb4b-4bea-aad2-d04651a42460 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:22.662293 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a839b8e-265c-4319-95b9-09929041f5ee None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:22.677051 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:22.679112 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97184ab6-bb4b-4bea-aad2-d04651a42460 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:22.681356 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97184ab6-bb4b-4bea-aad2-d04651a42460 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:22.719829 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3400ca38-6e83-48f2-87b2-0e2ab647a751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:22.720534 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3400ca38-6e83-48f2-87b2-0e2ab647a751 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:22.720828 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3400ca38-6e83-48f2-87b2-0e2ab647a751 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:22.721108 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3400ca38-6e83-48f2-87b2-0e2ab647a751 None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:22.721845 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3400ca38-6e83-48f2-87b2-0e2ab647a751 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:22.722420 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3400ca38-6e83-48f2-87b2-0e2ab647a751 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:22.724806 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3400ca38-6e83-48f2-87b2-0e2ab647a751 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:22.737076 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 356/689] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:56:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:22.738691 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77cf68-bae7-44c4-b013-8a092a97a966 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:22.738983 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77cf68-bae7-44c4-b013-8a092a97a966 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:22.739255 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77cf68-bae7-44c4-b013-8a092a97a966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:22.761837 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-9c77cf68-bae7-44c4-b013-8a092a97a966 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:22.762975 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-9c77cf68-bae7-44c4-b013-8a092a97a966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:22.809237 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 334/690] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:56:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:56:22.810221 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97184ab6-bb4b-4bea-aad2-d04651a42460 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:22.817751 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:22.839971 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 357/691] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:56:22.840865 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c77cf68-bae7-44c4-b013-8a092a97a966 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:22.927052 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97184ab6-bb4b-4bea-aad2-d04651a42460 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:22.967388 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ded6b2-812d-4ad2-b9d6-4b98ea955366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:22.969521 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ded6b2-812d-4ad2-b9d6-4b98ea955366 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:22.970032 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ded6b2-812d-4ad2-b9d6-4b98ea955366 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:22.970454 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ded6b2-812d-4ad2-b9d6-4b98ea955366 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:22.982128 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ded6b2-812d-4ad2-b9d6-4b98ea955366 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:22.983847 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ded6b2-812d-4ad2-b9d6-4b98ea955366 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:22.985483 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:22.989645 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c77cf68-bae7-44c4-b013-8a092a97a966 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:23.003036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-50ded6b2-812d-4ad2-b9d6-4b98ea955366 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:23.003036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:23.003036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:23.003036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:23.003036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:23.003036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:23.003036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:23.003036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:23.003036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:23.003036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:23.003036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:23.003036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:23.003036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:23.003036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:23.003036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:23.004007 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:23.004007 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:23.004007 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:23.004007 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:23.004007 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:23.004007 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:23.004007 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:23.004007 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:23.004007 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:23.004007 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:23.004007 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:23.004007 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:23.004007 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:23.004007 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:23.004007 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:23.004007 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:23.004007 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:23.004973 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:23.004973 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:23.004973 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:23.004973 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:23.012739 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 335/692] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:23.013954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ded6b2-812d-4ad2-b9d6-4b98ea955366 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:23.035169 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c9cbc-d35e-4939-85ac-4cf44f3b2cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:23.036285 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c9cbc-d35e-4939-85ac-4cf44f3b2cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:23.036655 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c9cbc-d35e-4939-85ac-4cf44f3b2cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:23.036998 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c9cbc-d35e-4939-85ac-4cf44f3b2cf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:23.050446 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c9cbc-d35e-4939-85ac-4cf44f3b2cf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:23.052277 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c9cbc-d35e-4939-85ac-4cf44f3b2cf7 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:23.072394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-e08c9cbc-d35e-4939-85ac-4cf44f3b2cf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:23.072394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:23.072394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:23.072394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:23.072394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:23.072394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:23.072394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:23.072394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:23.072394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:23.072394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:23.072394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:23.072394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:23.072394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:23.072394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:23.072394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:23.073456 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:23.073456 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:23.073456 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:23.073456 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:23.073456 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:23.073456 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:23.073456 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:23.073456 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:23.073456 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:23.073456 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:23.073456 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:23.073456 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:23.073456 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:23.073456 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:23.073456 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:23.073456 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:23.073456 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:23.074352 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:23.074352 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:23.074352 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:23.074352 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:23.074352 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 358/693] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:23.075170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c9cbc-d35e-4939-85ac-4cf44f3b2cf7 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:23.083181 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:23.139075 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ded6b2-812d-4ad2-b9d6-4b98ea955366 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:23.185469 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af95d8e-eaf1-4813-a144-88df09dd47b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:23.186220 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af95d8e-eaf1-4813-a144-88df09dd47b8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:23.186704 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af95d8e-eaf1-4813-a144-88df09dd47b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:23.187078 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af95d8e-eaf1-4813-a144-88df09dd47b8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:23.188185 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af95d8e-eaf1-4813-a144-88df09dd47b8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:23.189848 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af95d8e-eaf1-4813-a144-88df09dd47b8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:23.196465 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c9cbc-d35e-4939-85ac-4cf44f3b2cf7 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:23.204510 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-0af95d8e-eaf1-4813-a144-88df09dd47b8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 13:56:23.204510 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:23.204510 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:56:23.204510 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:56:23.204510 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:56:23.204510 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:56:23.204510 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:56:23.204510 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:23.204510 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 13:56:23.204510 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:23.204510 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:56:23.204510 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:23.204510 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:23.204510 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:23.204510 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:23.204510 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:23.205545 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:23.205545 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:23.205545 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:23.205545 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:23.205545 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:23.205545 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:23.205545 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:23.205545 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:56:23.205545 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:56:23.205545 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:56:23.205545 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:56:23.205545 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:23.205545 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:23.205545 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:23.205545 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:23.205545 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:23.206393 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:23.206393 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:23.206393 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:23.206393 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:23.206393 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:23.206393 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:23.206393 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:23.206393 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:23.206393 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:23.206393 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:23.206393 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:23.206393 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:56:23.206393 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:23.206393 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 13:56:23.206393 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:23.208568 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 336/694] 10.222.0.48 () {56 vars in 1129 bytes} [Mon Aug 1 13:56:23 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:23.209508 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af95d8e-eaf1-4813-a144-88df09dd47b8 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:23.235441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48043fb5-e2d3-447c-bf8f-a81a590ade52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:23.236263 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48043fb5-e2d3-447c-bf8f-a81a590ade52 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:23.236685 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48043fb5-e2d3-447c-bf8f-a81a590ade52 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:23.237067 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48043fb5-e2d3-447c-bf8f-a81a590ade52 None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:23.237873 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48043fb5-e2d3-447c-bf8f-a81a590ade52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:23.238285 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48043fb5-e2d3-447c-bf8f-a81a590ade52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:23.240473 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48043fb5-e2d3-447c-bf8f-a81a590ade52 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:23.254026 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 359/695] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:23.255046 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48043fb5-e2d3-447c-bf8f-a81a590ade52 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:23.280906 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:23.334214 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af95d8e-eaf1-4813-a144-88df09dd47b8 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:23.352021 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:23.362396 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48043fb5-e2d3-447c-bf8f-a81a590ade52 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:23.370503 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f900f-5371-49ae-af49-95c30ccb8dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:23.371359 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f900f-5371-49ae-af49-95c30ccb8dad None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:23.371817 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f900f-5371-49ae-af49-95c30ccb8dad None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:23.372268 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f900f-5371-49ae-af49-95c30ccb8dad None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:23.373209 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f900f-5371-49ae-af49-95c30ccb8dad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:23.373713 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f900f-5371-49ae-af49-95c30ccb8dad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:23.375271 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f900f-5371-49ae-af49-95c30ccb8dad None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:23.388075 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 337/696] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:23.389137 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f900f-5371-49ae-af49-95c30ccb8dad None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:23.398759 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a82354-c876-485e-a181-bc888d952bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:23.399444 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a82354-c876-485e-a181-bc888d952bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:23.399729 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a82354-c876-485e-a181-bc888d952bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:23.400007 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a82354-c876-485e-a181-bc888d952bd5 None admin] PATH_INFO: `/v3/services` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:23.400819 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a82354-c876-485e-a181-bc888d952bd5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:23.401129 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a82354-c876-485e-a181-bc888d952bd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:23.402671 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a82354-c876-485e-a181-bc888d952bd5 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:23.411763 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 360/697] 10.222.0.48 () {56 vars in 1124 bytes} [Mon Aug 1 13:56:23 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:23.412687 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a82354-c876-485e-a181-bc888d952bd5 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:23.492829 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:23.495619 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f900f-5371-49ae-af49-95c30ccb8dad None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:23.519245 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a82354-c876-485e-a181-bc888d952bd5 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:23.530696 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91ba9c-12f8-47f7-b3ec-0d4a3248cfeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:23.531338 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91ba9c-12f8-47f7-b3ec-0d4a3248cfeb None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:23.531570 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91ba9c-12f8-47f7-b3ec-0d4a3248cfeb None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:23.531847 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91ba9c-12f8-47f7-b3ec-0d4a3248cfeb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:23.558491 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35045fc6-cd8b-4418-a027-1e4c5e1febd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:23.559589 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35045fc6-cd8b-4418-a027-1e4c5e1febd4 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:23.560105 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35045fc6-cd8b-4418-a027-1e4c5e1febd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:23.560653 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35045fc6-cd8b-4418-a027-1e4c5e1febd4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:23.569284 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35045fc6-cd8b-4418-a027-1e4c5e1febd4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:23.571310 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35045fc6-cd8b-4418-a027-1e4c5e1febd4 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:23.584814 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-35045fc6-cd8b-4418-a027-1e4c5e1febd4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:23.584814 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:23.584814 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:23.584814 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:23.584814 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:23.584814 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:23.584814 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:23.584814 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:23.584814 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:23.584814 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:23.584814 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:23.584814 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:23.584814 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:56:23.584814 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:56:23.584814 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:56:23.585872 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:56:23.585872 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:23.585872 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:23.585872 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:23.585872 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:23.585872 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:23.585872 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:23.585872 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:23.585872 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:23.585872 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:23.585872 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:23.585872 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:23.585872 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:23.585872 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:23.585872 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:23.585872 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:23.585872 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:23.586766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:56:23.586766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:56:23.586766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:56:23.586766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:56:23.586766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:56:23.586766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:56:23.586766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:23.586766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:23.589475 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 361/698] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:56:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:23.590747 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35045fc6-cd8b-4418-a027-1e4c5e1febd4 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:23.655899 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a91ba9c-12f8-47f7-b3ec-0d4a3248cfeb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:23.657381 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a91ba9c-12f8-47f7-b3ec-0d4a3248cfeb None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:23.670101 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-1a91ba9c-12f8-47f7-b3ec-0d4a3248cfeb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 13:56:23.670101 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:23.670101 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:23.670101 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:23.670101 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:23.670101 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:23.670101 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:23.670101 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:23.670101 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:23.670101 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:23.670101 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:23.670101 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:23.670101 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:56:23.670101 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:56:23.670101 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:56:23.670101 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:56:23.671100 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:23.671100 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:23.671100 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:56:23.671100 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:23.671100 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:56:23.671100 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:23.671100 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:23.671100 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:23.671100 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:23.671100 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:23.671100 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:23.671100 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:23.671100 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:23.671100 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:23.671100 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:23.671100 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:23.671100 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:23.671100 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:23.672626 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:23.672626 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:23.672626 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:56:23.672626 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:56:23.672626 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:56:23.672626 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:56:23.672626 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:56:23.672626 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:56:23.672626 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 13:56:23.672626 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:23.672626 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 338/699] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Aug 1 13:56:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:23.672626 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91ba9c-12f8-47f7-b3ec-0d4a3248cfeb None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:23.720007 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:23.789756 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91ba9c-12f8-47f7-b3ec-0d4a3248cfeb None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:23.825345 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d659766-6672-49ff-8365-2ef968dae126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:23.825943 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d659766-6672-49ff-8365-2ef968dae126 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:23.826220 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d659766-6672-49ff-8365-2ef968dae126 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:23.826500 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d659766-6672-49ff-8365-2ef968dae126 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:23.827246 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d659766-6672-49ff-8365-2ef968dae126 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:23.827552 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d659766-6672-49ff-8365-2ef968dae126 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:23.828991 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d659766-6672-49ff-8365-2ef968dae126 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:23.835935 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35045fc6-cd8b-4418-a027-1e4c5e1febd4 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:23.841713 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 339/700] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 13:56:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:23.842543 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d659766-6672-49ff-8365-2ef968dae126 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:23.874837 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0250304-a500-4b69-9dc9-ecdd38934e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:23.875871 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0250304-a500-4b69-9dc9-ecdd38934e81 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:23.876357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0250304-a500-4b69-9dc9-ecdd38934e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:23.876912 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0250304-a500-4b69-9dc9-ecdd38934e81 None admin] PATH_INFO: `/v3/services` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:23.877894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0250304-a500-4b69-9dc9-ecdd38934e81 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:23.878448 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0250304-a500-4b69-9dc9-ecdd38934e81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:23.880195 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0250304-a500-4b69-9dc9-ecdd38934e81 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:23.888811 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:23.892779 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 362/701] 10.222.0.48 () {56 vars in 1124 bytes} [Mon Aug 1 13:56:23 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:23.893958 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0250304-a500-4b69-9dc9-ecdd38934e81 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:23.944585 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:23.959851 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d659766-6672-49ff-8365-2ef968dae126 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:24.019656 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9cdf1-773e-4755-9cbb-4a303431c990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:24.019656 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9cdf1-773e-4755-9cbb-4a303431c990 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:24.019656 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9cdf1-773e-4755-9cbb-4a303431c990 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:24.019656 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9cdf1-773e-4755-9cbb-4a303431c990 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:24.020414 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9cdf1-773e-4755-9cbb-4a303431c990 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:24.020749 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9cdf1-773e-4755-9cbb-4a303431c990 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:24.022178 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9cdf1-773e-4755-9cbb-4a303431c990 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:24.043161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 340/702] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Aug 1 13:56:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:24.048094 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9cdf1-773e-4755-9cbb-4a303431c990 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:24.063325 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0250304-a500-4b69-9dc9-ecdd38934e81 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:24.137191 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75cfa48-8851-4c6f-8cda-6404b3f5ad9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:24.138209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75cfa48-8851-4c6f-8cda-6404b3f5ad9c None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:24.138688 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75cfa48-8851-4c6f-8cda-6404b3f5ad9c None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:24.139212 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75cfa48-8851-4c6f-8cda-6404b3f5ad9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:24.140220 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75cfa48-8851-4c6f-8cda-6404b3f5ad9c None admin] RBAC: Adding query filter params (role.id=d3d8169e71454005a8cf0d7544ad05dc, scope.project.id=72ecec9fad964b1fbeaefd0519597229, user.id=2fe2a6d7bd10435ba2c8e552d808b7a0) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:24.140553 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75cfa48-8851-4c6f-8cda-6404b3f5ad9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:24.142314 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75cfa48-8851-4c6f-8cda-6404b3f5ad9c None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:24.149549 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 363/703] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:56:23 2022] GET /identity/v3/role_assignments?user.id=2fe2a6d7bd10435ba2c8e552d808b7a0&scope.project.id=72ecec9fad964b1fbeaefd0519597229&role.id=d3d8169e71454005a8cf0d7544ad05dc => generated 608 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:24.151091 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac30aca-9e2a-416a-8924-523cedd23f61 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:24.151091 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac30aca-9e2a-416a-8924-523cedd23f61 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:24.151282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac30aca-9e2a-416a-8924-523cedd23f61 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:24.153578 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 364/704] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:24.164339 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c1ffb-ed58-43f4-b5b5-718f93bef35c None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:24.164847 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c1ffb-ed58-43f4-b5b5-718f93bef35c None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:24.165232 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c1ffb-ed58-43f4-b5b5-718f93bef35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:24.188907 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-d61c1ffb-ed58-43f4-b5b5-718f93bef35c None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:24.190371 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-d61c1ffb-ed58-43f4-b5b5-718f93bef35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:24.227207 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9cdf1-773e-4755-9cbb-4a303431c990 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:24.256332 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:24.266860 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 365/705] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:56:24.281326 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0718ab-4757-458d-bcbf-935020f8804d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:24.281978 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0718ab-4757-458d-bcbf-935020f8804d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:24.282270 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0718ab-4757-458d-bcbf-935020f8804d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:24.282552 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0718ab-4757-458d-bcbf-935020f8804d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:24.293552 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c1ffb-ed58-43f4-b5b5-718f93bef35c None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:24.315283 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:24.364113 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0718ab-4757-458d-bcbf-935020f8804d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:24.366150 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0718ab-4757-458d-bcbf-935020f8804d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:24.476776 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 341/706] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:56:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:56:24.478206 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0718ab-4757-458d-bcbf-935020f8804d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:24.490089 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c1ffb-ed58-43f4-b5b5-718f93bef35c None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:24.528539 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf551e98-7333-44e6-8074-2a749a10b60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:24.529692 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf551e98-7333-44e6-8074-2a749a10b60e None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:24.530177 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf551e98-7333-44e6-8074-2a749a10b60e None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:24.530647 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf551e98-7333-44e6-8074-2a749a10b60e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:24.536165 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:24.545356 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf551e98-7333-44e6-8074-2a749a10b60e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:24.547266 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf551e98-7333-44e6-8074-2a749a10b60e None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:24.564384 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-cf551e98-7333-44e6-8074-2a749a10b60e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:24.564384 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:24.564384 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:24.564384 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:24.564384 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:24.564384 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:24.564384 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:24.564384 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:24.564384 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:24.564384 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:24.564384 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:24.564384 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:24.564384 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:24.564384 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:24.564384 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:24.565313 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:24.565313 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:24.565313 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:24.565313 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:24.565313 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:24.565313 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:24.565313 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:24.565313 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:24.565313 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:24.565313 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:24.565313 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:24.565313 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:24.565313 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:24.565313 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:24.565313 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:24.565313 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:24.565313 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:24.566167 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:24.566167 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:24.566167 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:24.566167 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:24.568542 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 366/707] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:24.575416 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf551e98-7333-44e6-8074-2a749a10b60e None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:24.583660 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0718ab-4757-458d-bcbf-935020f8804d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:24.621616 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6389c5f6-345e-4e0d-a18b-b1836c3255cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:24.624783 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389c5f6-345e-4e0d-a18b-b1836c3255cc None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:24.624783 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389c5f6-345e-4e0d-a18b-b1836c3255cc None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:24.624783 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389c5f6-345e-4e0d-a18b-b1836c3255cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:24.636059 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:24.641914 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6389c5f6-345e-4e0d-a18b-b1836c3255cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:24.643783 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6389c5f6-345e-4e0d-a18b-b1836c3255cc None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:24.667028 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-6389c5f6-345e-4e0d-a18b-b1836c3255cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:24.667028 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:24.667028 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:24.667028 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:24.667028 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:24.667028 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:24.667028 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:24.667028 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:24.667028 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:24.667028 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:24.667028 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:24.667028 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:24.667028 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:24.667028 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:24.667028 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:24.667963 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:24.667963 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:24.667963 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:24.667963 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:24.667963 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:24.667963 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:24.667963 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:24.667963 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:24.667963 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:24.667963 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:24.667963 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:24.667963 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:24.667963 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:24.667963 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:24.667963 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:24.667963 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:24.667963 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:24.668846 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:24.668846 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:24.668846 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:24.668846 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:24.676077 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 342/708] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:56:24.685959 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6389c5f6-345e-4e0d-a18b-b1836c3255cc None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:24.725218 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf551e98-7333-44e6-8074-2a749a10b60e None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:24.765593 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff15643-c200-41b4-bd8a-517a8fc8ee3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:24.766370 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff15643-c200-41b4-bd8a-517a8fc8ee3a None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:24.766747 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff15643-c200-41b4-bd8a-517a8fc8ee3a None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:24.767117 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff15643-c200-41b4-bd8a-517a8fc8ee3a None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:24.767910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff15643-c200-41b4-bd8a-517a8fc8ee3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:24.768312 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff15643-c200-41b4-bd8a-517a8fc8ee3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:24.769812 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff15643-c200-41b4-bd8a-517a8fc8ee3a None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:24.783493 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 367/709] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:56:24.785067 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b4d02-fe04-4142-a174-b24cc26b26f2 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:24.785450 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b4d02-fe04-4142-a174-b24cc26b26f2 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:24.785865 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b4d02-fe04-4142-a174-b24cc26b26f2 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:24.787399 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 368/710] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:24.790349 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1b4d02-fe04-4142-a174-b24cc26b26f2 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:24.842861 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6389c5f6-345e-4e0d-a18b-b1836c3255cc None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:24.855493 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:24.891746 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb07f68-c065-4907-ba7c-556585c7bd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:24.892698 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb07f68-c065-4907-ba7c-556585c7bd77 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:24.893932 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb07f68-c065-4907-ba7c-556585c7bd77 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:24.894143 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb07f68-c065-4907-ba7c-556585c7bd77 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:24.894837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb07f68-c065-4907-ba7c-556585c7bd77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:24.895146 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb07f68-c065-4907-ba7c-556585c7bd77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:24.896565 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb07f68-c065-4907-ba7c-556585c7bd77 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:24.914696 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 343/711] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:24.915895 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b9d5b-9825-47fe-adf3-6fddeb1ffb1b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:24.916184 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b9d5b-9825-47fe-adf3-6fddeb1ffb1b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:24.916460 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b9d5b-9825-47fe-adf3-6fddeb1ffb1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:24.921020 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1b4d02-fe04-4142-a174-b24cc26b26f2 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:24.930507 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:24.935008 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-aa9b9d5b-9825-47fe-adf3-6fddeb1ffb1b None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:24.936451 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-aa9b9d5b-9825-47fe-adf3-6fddeb1ffb1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:24.972228 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a568fc7-5058-4ffe-9ff2-3706ae7c254c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:24.972900 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a568fc7-5058-4ffe-9ff2-3706ae7c254c None admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:24.973188 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a568fc7-5058-4ffe-9ff2-3706ae7c254c None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:24.973469 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a568fc7-5058-4ffe-9ff2-3706ae7c254c None admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:24.974619 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a568fc7-5058-4ffe-9ff2-3706ae7c254c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:24.976927 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a568fc7-5058-4ffe-9ff2-3706ae7c254c None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:25.012521 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 369/712] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 13:56:24 2022] POST /identity/v3/users => generated 254 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:56:25.012521 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a568fc7-5058-4ffe-9ff2-3706ae7c254c None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:25.071206 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 344/713] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:56:25.118452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b9d5b-9825-47fe-adf3-6fddeb1ffb1b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:25.215767 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a568fc7-5058-4ffe-9ff2-3706ae7c254c None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:25.254812 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c8493-54f5-4223-afba-0605a6d1588a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:25.256249 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c8493-54f5-4223-afba-0605a6d1588a None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:25.256824 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c8493-54f5-4223-afba-0605a6d1588a None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:25.257723 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c8493-54f5-4223-afba-0605a6d1588a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:25.258741 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:25.263423 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b9d5b-9825-47fe-adf3-6fddeb1ffb1b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:25.268608 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898c8493-54f5-4223-afba-0605a6d1588a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:25.270401 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898c8493-54f5-4223-afba-0605a6d1588a None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:25.283548 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-898c8493-54f5-4223-afba-0605a6d1588a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:25.283548 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:25.283548 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:25.283548 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:25.283548 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:25.283548 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:25.283548 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:25.283548 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:25.283548 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:25.283548 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:25.283548 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:25.283548 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:25.283548 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:56:25.283548 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:56:25.283548 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:56:25.284535 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:56:25.284535 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:25.284535 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:25.284535 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:25.284535 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:25.284535 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:25.284535 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:25.284535 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:25.284535 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:25.284535 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:25.284535 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:25.284535 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:25.284535 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:25.284535 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:25.284535 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:25.284535 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:25.284535 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:25.285418 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:56:25.285418 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:56:25.285418 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:56:25.285418 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:56:25.285418 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:56:25.285418 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:56:25.285418 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:25.285418 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:25.285418 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 370/714] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:56:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:25.290862 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c8493-54f5-4223-afba-0605a6d1588a None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:25.303282 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ea421-ed43-46d9-8e37-c34000bf5d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:25.304886 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ea421-ed43-46d9-8e37-c34000bf5d62 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:25.305185 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ea421-ed43-46d9-8e37-c34000bf5d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:25.305765 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ea421-ed43-46d9-8e37-c34000bf5d62 None admin] PATH_INFO: `/v3/services/network` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:25.306426 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352ea421-ed43-46d9-8e37-c34000bf5d62 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:25.308023 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352ea421-ed43-46d9-8e37-c34000bf5d62 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:25.320993 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:25.321805 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-352ea421-ed43-46d9-8e37-c34000bf5d62 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 13:56:25.321805 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:25.321805 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:56:25.321805 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:56:25.321805 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:56:25.321805 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:56:25.321805 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:56:25.321805 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:25.321805 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 13:56:25.321805 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:25.321805 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:56:25.321805 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:25.321805 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:25.321805 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:25.321805 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:25.321805 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:25.321805 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:25.322730 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:25.322730 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:25.322730 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:25.322730 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:25.322730 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:25.322730 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:25.322730 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:56:25.322730 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:56:25.322730 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:56:25.322730 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:56:25.322730 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:25.322730 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:25.322730 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:25.322730 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:25.322730 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:25.322730 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:25.323539 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:25.323539 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:25.323539 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:25.323539 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:25.323539 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:25.323539 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:25.323539 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:25.323539 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:25.323539 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:25.323539 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:25.323539 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:56:25.323539 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:25.323539 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 13:56:25.323539 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:25.323539 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 345/715] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:56:25 2022] GET /identity/v3/services/network => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:56:25.331246 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ea421-ed43-46d9-8e37-c34000bf5d62 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:25.426304 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c8493-54f5-4223-afba-0605a6d1588a None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:25.471234 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ea421-ed43-46d9-8e37-c34000bf5d62 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:25.472149 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f59455b-87e8-4f33-badf-eda20e15551e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:25.473260 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59455b-87e8-4f33-badf-eda20e15551e None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:25.474591 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59455b-87e8-4f33-badf-eda20e15551e None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:25.475095 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59455b-87e8-4f33-badf-eda20e15551e None admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:25.476685 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59455b-87e8-4f33-badf-eda20e15551e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:25.477175 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59455b-87e8-4f33-badf-eda20e15551e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:25.478772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59455b-87e8-4f33-badf-eda20e15551e None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:25.495863 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 371/716] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 13:56:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:56:25.501689 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f59455b-87e8-4f33-badf-eda20e15551e None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:25.513168 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c9494-6afa-48be-87c0-1e63e84f7f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:25.513860 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c9494-6afa-48be-87c0-1e63e84f7f29 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:25.514150 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c9494-6afa-48be-87c0-1e63e84f7f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:25.514429 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c9494-6afa-48be-87c0-1e63e84f7f29 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:25.515900 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350c9494-6afa-48be-87c0-1e63e84f7f29 None admin] RBAC: Adding query filter params (name=network) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:25.516219 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350c9494-6afa-48be-87c0-1e63e84f7f29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:25.517714 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350c9494-6afa-48be-87c0-1e63e84f7f29 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:25.525484 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 346/717] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 13:56:25 2022] GET /identity/v3/services?name=network => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:25.531271 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c9494-6afa-48be-87c0-1e63e84f7f29 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:25.564323 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:25.614713 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:25.665143 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c9494-6afa-48be-87c0-1e63e84f7f29 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:25.673613 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f59455b-87e8-4f33-badf-eda20e15551e None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:25.702548 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5b1eef-f92b-4a0c-820d-eb0a0b8d0c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:25.703191 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b1eef-f92b-4a0c-820d-eb0a0b8d0c41 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:25.703478 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b1eef-f92b-4a0c-820d-eb0a0b8d0c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:25.703755 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b1eef-f92b-4a0c-820d-eb0a0b8d0c41 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:25.704452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5b1eef-f92b-4a0c-820d-eb0a0b8d0c41 None admin] RBAC: Adding query filter params (type=network) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:25.704805 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5b1eef-f92b-4a0c-820d-eb0a0b8d0c41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:25.706202 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5b1eef-f92b-4a0c-820d-eb0a0b8d0c41 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:25.723140 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 347/718] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 13:56:25 2022] GET /identity/v3/services?type=network => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:25.741250 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14405d28-75a4-4bb6-a257-045462f8258d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:25.741896 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14405d28-75a4-4bb6-a257-045462f8258d None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:25.742182 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14405d28-75a4-4bb6-a257-045462f8258d None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:25.742457 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14405d28-75a4-4bb6-a257-045462f8258d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:25.743268 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14405d28-75a4-4bb6-a257-045462f8258d None admin] RBAC: Adding query filter params (role.id=58f91e3a13664dabb18c6227a1658fa6, scope.project.id=72ecec9fad964b1fbeaefd0519597229, user.id=a246654c783b4cc68d212e91bb05d8c6) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:25.743577 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14405d28-75a4-4bb6-a257-045462f8258d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:25.745077 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14405d28-75a4-4bb6-a257-045462f8258d None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:25.758343 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 372/719] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:56:25 2022] GET /identity/v3/role_assignments?user.id=a246654c783b4cc68d212e91bb05d8c6&scope.project.id=72ecec9fad964b1fbeaefd0519597229&role.id=58f91e3a13664dabb18c6227a1658fa6 => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:26.424231 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4263be-b893-42e8-a5d4-b8c59930a41d None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:26.424770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4263be-b893-42e8-a5d4-b8c59930a41d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:26.424847 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4263be-b893-42e8-a5d4-b8c59930a41d None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:26.426522 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 348/720] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:26.431538 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90336b-d2ef-47d2-9ca3-1e1a461f2b1a None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:26.431924 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90336b-d2ef-47d2-9ca3-1e1a461f2b1a None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:26.432112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90336b-d2ef-47d2-9ca3-1e1a461f2b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:26.461248 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-bf90336b-d2ef-47d2-9ca3-1e1a461f2b1a None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:26.462463 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-bf90336b-d2ef-47d2-9ca3-1e1a461f2b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:26.561667 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 373/721] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:56:26.596666 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4263be-b893-42e8-a5d4-b8c59930a41d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:26.662208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831cb30-b8d9-42c8-9286-d1ada6d9bb87 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:26.662572 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831cb30-b8d9-42c8-9286-d1ada6d9bb87 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:26.662783 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831cb30-b8d9-42c8-9286-d1ada6d9bb87 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:26.664433 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 374/722] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:26.669369 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92c1c3-a9a1-42d0-80c1-482d9a506e5a None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:26.669663 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92c1c3-a9a1-42d0-80c1-482d9a506e5a None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:26.669982 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92c1c3-a9a1-42d0-80c1-482d9a506e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:26.698560 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-5a92c1c3-a9a1-42d0-80c1-482d9a506e5a None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:26.702524 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-5a92c1c3-a9a1-42d0-80c1-482d9a506e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:26.745870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:26.767072 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4263be-b893-42e8-a5d4-b8c59930a41d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:26.830274 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a3f40-4b8f-4e3d-9d7a-a6371b9b3ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:26.831335 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a3f40-4b8f-4e3d-9d7a-a6371b9b3ad7 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:26.831870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a3f40-4b8f-4e3d-9d7a-a6371b9b3ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:26.832167 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a3f40-4b8f-4e3d-9d7a-a6371b9b3ad7 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:26.832924 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a3f40-4b8f-4e3d-9d7a-a6371b9b3ad7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:26.834299 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a3f40-4b8f-4e3d-9d7a-a6371b9b3ad7 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:26.849412 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 375/723] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 13:56:26.879646 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 349/724] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 13:56:26 2022] POST /identity/v3/services => generated 251 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:56:26.890762 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92c1c3-a9a1-42d0-80c1-482d9a506e5a None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:26.901820 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:27.037236 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92c1c3-a9a1-42d0-80c1-482d9a506e5a None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:27.089485 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70c51b-bb4e-4d40-94a7-11f17813e92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:27.090179 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70c51b-bb4e-4d40-94a7-11f17813e92c None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:27.090435 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70c51b-bb4e-4d40-94a7-11f17813e92c None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:27.090778 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70c51b-bb4e-4d40-94a7-11f17813e92c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:27.102097 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f70c51b-bb4e-4d40-94a7-11f17813e92c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:56:27.107042 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-6f70c51b-bb4e-4d40-94a7-11f17813e92c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:56:27.107042 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:27.107042 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:27.107042 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:27.107042 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:27.107042 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:27.107042 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:27.107042 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:27.107042 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:27.107042 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:27.107042 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:27.107042 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:27.107042 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:56:27.107042 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:56:27.107042 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:56:27.107042 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:56:27.107962 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:56:27.107962 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:56:27.107962 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:56:27.107962 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:56:27.107962 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:27.107962 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:27.107962 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:27.107962 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:27.107962 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:27.107962 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:27.107962 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:27.107962 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:27.107962 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:27.107962 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:27.107962 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:27.107962 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:27.108797 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:27.108797 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:27.108797 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:27.108797 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:27.108797 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:56:27.108797 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:56:27.108797 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:56:27.108797 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:56:27.108797 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:56:27.108797 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:56:27.108797 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:56:27.108797 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:27.112626 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 376/725] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 13:56:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:27.114514 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a3f40-4b8f-4e3d-9d7a-a6371b9b3ad7 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:27.234773 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a3f40-4b8f-4e3d-9d7a-a6371b9b3ad7 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:27.273530 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b73fc5a-e4cb-4962-a3e3-1bfb6658017f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:27.274165 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73fc5a-e4cb-4962-a3e3-1bfb6658017f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:27.274452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73fc5a-e4cb-4962-a3e3-1bfb6658017f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:27.274793 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73fc5a-e4cb-4962-a3e3-1bfb6658017f None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:27.275527 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b73fc5a-e4cb-4962-a3e3-1bfb6658017f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:27.275839 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b73fc5a-e4cb-4962-a3e3-1bfb6658017f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:27.277242 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b73fc5a-e4cb-4962-a3e3-1bfb6658017f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:27.286294 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 350/726] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:56:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:27.292836 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70c51b-bb4e-4d40-94a7-11f17813e92c None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:27.364920 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:27.405564 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70c51b-bb4e-4d40-94a7-11f17813e92c None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:27.441048 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a6ed9-eb34-4617-82ec-0b667588a7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:27.441806 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a6ed9-eb34-4617-82ec-0b667588a7dd None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:27.441991 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a6ed9-eb34-4617-82ec-0b667588a7dd None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:27.442268 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a6ed9-eb34-4617-82ec-0b667588a7dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:27.453332 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a6ed9-eb34-4617-82ec-0b667588a7dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:27.454786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a6ed9-eb34-4617-82ec-0b667588a7dd None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:27.470112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-e19a6ed9-eb34-4617-82ec-0b667588a7dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:27.470112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:27.470112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:27.470112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:27.470112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:27.470112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:27.470112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:27.470112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:27.470112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:27.470112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:27.470112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:27.470112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:27.470112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:27.470112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:27.470112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:27.471012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:27.471012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:27.471012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:27.471012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:27.471012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:27.471012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:27.471012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:27.471012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:27.471012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:27.471012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:27.471012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:27.471012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:27.471012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:27.471012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:27.471012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:27.471012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:27.471012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:27.471883 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:27.471883 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:27.471883 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:27.471883 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:27.471883 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 377/727] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:27.477353 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b73fc5a-e4cb-4962-a3e3-1bfb6658017f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:27.494081 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:27.592807 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b73fc5a-e4cb-4962-a3e3-1bfb6658017f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:27.628102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9382ba0-4fa0-43cd-84ed-5b06a392992c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:27.628819 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9382ba0-4fa0-43cd-84ed-5b06a392992c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:27.629116 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9382ba0-4fa0-43cd-84ed-5b06a392992c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:27.629399 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9382ba0-4fa0-43cd-84ed-5b06a392992c None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:27.630114 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9382ba0-4fa0-43cd-84ed-5b06a392992c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:27.630427 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9382ba0-4fa0-43cd-84ed-5b06a392992c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:27.632649 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9382ba0-4fa0-43cd-84ed-5b06a392992c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:27.645388 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 351/728] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:27.650914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a6ed9-eb34-4617-82ec-0b667588a7dd None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:27.765606 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a6ed9-eb34-4617-82ec-0b667588a7dd None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:27.783492 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:27.817214 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e1d94-af14-4e1f-b77b-d3bcabb24f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:27.818322 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e1d94-af14-4e1f-b77b-d3bcabb24f35 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:27.818806 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e1d94-af14-4e1f-b77b-d3bcabb24f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:27.819357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e1d94-af14-4e1f-b77b-d3bcabb24f35 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:27.833578 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e1d94-af14-4e1f-b77b-d3bcabb24f35 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:27.835419 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e1d94-af14-4e1f-b77b-d3bcabb24f35 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:27.849423 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9dcb2b-7fe5-4fac-9e12-4e8d283235d5 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:27.849680 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9dcb2b-7fe5-4fac-9e12-4e8d283235d5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:27.849923 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9dcb2b-7fe5-4fac-9e12-4e8d283235d5 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:27.851221 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-550e1d94-af14-4e1f-b77b-d3bcabb24f35 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 13:56:27.851221 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:27.851221 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:27.851221 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:27.851221 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:27.851221 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:27.851221 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:27.851221 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:27.851221 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:27.851221 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:27.851221 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:27.851221 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:27.851221 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:56:27.851221 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:56:27.851221 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:56:27.851221 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:56:27.852080 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:27.852080 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:27.852080 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:56:27.852080 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:27.852080 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:56:27.852080 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:27.852080 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:27.852080 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:27.852080 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:27.852080 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:27.852080 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:27.852080 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:27.852080 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:27.852080 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:27.852080 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:27.852080 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:27.852080 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:27.852080 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:27.853022 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:27.853022 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:27.853022 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:56:27.853022 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:56:27.853022 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:56:27.853022 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:56:27.853022 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:56:27.853022 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:56:27.853022 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 13:56:27.853022 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:27.853541 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 352/729] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:27.855506 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 378/730] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 13:56:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:27.860199 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba0ecc-e2f5-4fb6-9a3b-8e092dd85791 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:27.860199 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba0ecc-e2f5-4fb6-9a3b-8e092dd85791 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:27.860199 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba0ecc-e2f5-4fb6-9a3b-8e092dd85791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:27.865665 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e1d94-af14-4e1f-b77b-d3bcabb24f35 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:27.880886 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a5ba0ecc-e2f5-4fb6-9a3b-8e092dd85791 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:27.883868 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a5ba0ecc-e2f5-4fb6-9a3b-8e092dd85791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:27.953243 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 353/731] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:27 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:56:27.985843 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba0ecc-e2f5-4fb6-9a3b-8e092dd85791 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:27.992786 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:28.043484 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e1d94-af14-4e1f-b77b-d3bcabb24f35 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:28.101335 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9dfca-8a7b-4639-82d4-25a94d9e4edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:28.103823 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9dfca-8a7b-4639-82d4-25a94d9e4edc None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:28.104108 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9dfca-8a7b-4639-82d4-25a94d9e4edc None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:28.105203 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9dfca-8a7b-4639-82d4-25a94d9e4edc None admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:28.106053 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d9dfca-8a7b-4639-82d4-25a94d9e4edc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:28.106367 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d9dfca-8a7b-4639-82d4-25a94d9e4edc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:28.107829 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d9dfca-8a7b-4639-82d4-25a94d9e4edc None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:28.144163 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 379/732] 10.222.0.48 () {56 vars in 1135 bytes} [Mon Aug 1 13:56:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:28.145843 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4fb787-6f0d-411c-bf30-ad7dec333f9d None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:28.146139 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4fb787-6f0d-411c-bf30-ad7dec333f9d None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:28.146420 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4fb787-6f0d-411c-bf30-ad7dec333f9d None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:28.152617 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 380/733] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:28.154543 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4fb787-6f0d-411c-bf30-ad7dec333f9d None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:28.167327 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba0ecc-e2f5-4fb6-9a3b-8e092dd85791 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:28.193682 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:28.212522 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003cefcf-6baa-4718-b1c1-f42d1ea7dd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:28.213608 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003cefcf-6baa-4718-b1c1-f42d1ea7dd13 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:28.214106 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003cefcf-6baa-4718-b1c1-f42d1ea7dd13 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:28.214931 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003cefcf-6baa-4718-b1c1-f42d1ea7dd13 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:28.227714 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003cefcf-6baa-4718-b1c1-f42d1ea7dd13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:56:28.234258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-003cefcf-6baa-4718-b1c1-f42d1ea7dd13 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:56:28.234258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:28.234258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:28.234258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:28.234258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:28.234258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:28.234258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:28.234258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:28.234258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:28.234258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:28.234258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:28.234258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:28.234258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:56:28.234258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:56:28.234258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:56:28.234258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:56:28.235173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:56:28.235173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:56:28.235173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:56:28.235173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:56:28.235173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:28.235173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:28.235173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:28.235173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:28.235173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:28.235173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:28.235173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:28.235173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:28.235173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:28.235173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:28.235173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:28.235173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:28.235976 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:28.235976 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:28.235976 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:28.235976 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:28.235976 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:56:28.235976 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:56:28.235976 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:56:28.235976 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:56:28.235976 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:56:28.235976 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:56:28.235976 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:56:28.235976 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:28.238032 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 354/734] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 13:56:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:56:28.239454 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bee809-03d6-4266-9437-ab9a32e259fe None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:28.239749 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bee809-03d6-4266-9437-ab9a32e259fe None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:28.240067 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bee809-03d6-4266-9437-ab9a32e259fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:28.260314 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-18bee809-03d6-4266-9437-ab9a32e259fe None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:28.261692 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-18bee809-03d6-4266-9437-ab9a32e259fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:28.282560 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4fb787-6f0d-411c-bf30-ad7dec333f9d None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:28.326784 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79119e43-614c-4682-b0d2-9264ecc193c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:28.327051 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:28.327903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79119e43-614c-4682-b0d2-9264ecc193c7 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:28.328525 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79119e43-614c-4682-b0d2-9264ecc193c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:28.328816 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79119e43-614c-4682-b0d2-9264ecc193c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:28.341395 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 355/735] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:28 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:56:28.342631 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bee809-03d6-4266-9437-ab9a32e259fe None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:28.421045 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79119e43-614c-4682-b0d2-9264ecc193c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:28.422867 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79119e43-614c-4682-b0d2-9264ecc193c7 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:28.457568 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:28.479828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bee809-03d6-4266-9437-ab9a32e259fe None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:28.519318 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545be0dd-569b-43d4-8764-656cc011c13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:28.520644 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545be0dd-569b-43d4-8764-656cc011c13e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:28.521164 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545be0dd-569b-43d4-8764-656cc011c13e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:28.521900 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545be0dd-569b-43d4-8764-656cc011c13e None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:28.523388 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545be0dd-569b-43d4-8764-656cc011c13e None admin] RBAC: Adding query filter params (name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:28.524157 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545be0dd-569b-43d4-8764-656cc011c13e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:28.526060 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545be0dd-569b-43d4-8764-656cc011c13e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:28.538405 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 356/736] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 13:56:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:28.539959 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545be0dd-569b-43d4-8764-656cc011c13e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:28.550662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 381/737] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:56:28 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 13:56:28.550662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79119e43-614c-4682-b0d2-9264ecc193c7 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:28.614159 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:28.660291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79119e43-614c-4682-b0d2-9264ecc193c7 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:28.665217 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545be0dd-569b-43d4-8764-656cc011c13e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:28.694707 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:28.701076 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30445e7-8b86-4f46-b4b9-bd74e3ef1a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:28.701717 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30445e7-8b86-4f46-b4b9-bd74e3ef1a6c None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:28.701998 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30445e7-8b86-4f46-b4b9-bd74e3ef1a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:28.702183 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb2fae0-6649-4a40-be25-175e281a1524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:28.702277 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30445e7-8b86-4f46-b4b9-bd74e3ef1a6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:28.702789 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2fae0-6649-4a40-be25-175e281a1524 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:28.703028 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2fae0-6649-4a40-be25-175e281a1524 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:28.703310 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2fae0-6649-4a40-be25-175e281a1524 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:28.713721 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb2fae0-6649-4a40-be25-175e281a1524 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:56:28.714144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30445e7-8b86-4f46-b4b9-bd74e3ef1a6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:28.715584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30445e7-8b86-4f46-b4b9-bd74e3ef1a6c None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:28.719068 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-2fb2fae0-6649-4a40-be25-175e281a1524 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:56:28.719068 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:28.719068 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:28.719068 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:28.719068 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:28.719068 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:28.719068 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:28.719068 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:28.719068 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:28.719068 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:28.719068 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:28.719068 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:28.719068 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:56:28.719068 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:56:28.719068 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:56:28.719068 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:56:28.719990 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:56:28.719990 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:56:28.719990 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:56:28.719990 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:56:28.719990 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:28.719990 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:28.719990 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:28.719990 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:28.719990 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:28.719990 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:28.719990 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:28.719990 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:28.719990 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:28.719990 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:28.719990 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:28.719990 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:28.720856 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:28.720856 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:28.720856 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:28.720856 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:28.720856 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:56:28.720856 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:56:28.720856 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:56:28.720856 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:56:28.720856 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:56:28.720856 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:56:28.720856 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:56:28.720856 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:28.725766 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 357/738] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 13:56:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:28.726342 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb2fae0-6649-4a40-be25-175e281a1524 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:28.736604 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-d30445e7-8b86-4f46-b4b9-bd74e3ef1a6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:28.736604 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:28.736604 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:28.736604 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:28.736604 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:28.736604 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:28.736604 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:28.736604 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:28.736604 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:28.736604 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:28.736604 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:28.736604 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:28.736604 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:28.736604 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:28.736604 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:28.738208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:28.738208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:28.738208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:28.738208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:28.738208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:28.738208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:28.738208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:28.738208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:28.738208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:28.738208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:28.738208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:28.738208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:28.738208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:28.738208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:28.738208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:28.738208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:28.738208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:28.739384 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:28.739384 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:28.739384 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:28.739384 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:28.743786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 382/739] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:28.745200 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967e6f5c-65d5-44da-8e66-20e993031496 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:28.745442 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967e6f5c-65d5-44da-8e66-20e993031496 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:28.745753 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967e6f5c-65d5-44da-8e66-20e993031496 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:28.751249 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 383/740] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:28.751776 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967e6f5c-65d5-44da-8e66-20e993031496 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:28.840943 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb2fae0-6649-4a40-be25-175e281a1524 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:28.856822 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:28.872409 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967e6f5c-65d5-44da-8e66-20e993031496 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:28.878229 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc092c-e787-4ab2-8598-b71e359538a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:28.878879 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc092c-e787-4ab2-8598-b71e359538a3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:28.879164 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc092c-e787-4ab2-8598-b71e359538a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:28.879444 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc092c-e787-4ab2-8598-b71e359538a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:28.891760 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc092c-e787-4ab2-8598-b71e359538a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:28.893237 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc092c-e787-4ab2-8598-b71e359538a3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:28.907555 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-4fdc092c-e787-4ab2-8598-b71e359538a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:28.907555 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:28.907555 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:28.907555 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:28.907555 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:28.907555 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:28.907555 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:28.907555 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:28.907555 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:28.907555 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:28.907555 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:28.907555 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:28.907555 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:28.907555 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:28.907555 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:28.908609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:28.908609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:28.908609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:28.908609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:28.908609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:28.908609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:28.908609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:28.908609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:28.908609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:28.908609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:28.908609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:28.908609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:28.908609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:28.908609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:28.908609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:28.908609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:28.908609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:28.909604 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:28.909604 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:28.909604 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:28.909604 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:28.909604 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 358/741] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:28.910585 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797ec4a-cc80-4826-9a0e-082c7d2ba624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:28.911368 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797ec4a-cc80-4826-9a0e-082c7d2ba624 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:28.911798 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797ec4a-cc80-4826-9a0e-082c7d2ba624 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:28.912210 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797ec4a-cc80-4826-9a0e-082c7d2ba624 None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:28.913133 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d797ec4a-cc80-4826-9a0e-082c7d2ba624 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:28.913488 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc092c-e787-4ab2-8598-b71e359538a3 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:28.913635 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d797ec4a-cc80-4826-9a0e-082c7d2ba624 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:28.914925 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d797ec4a-cc80-4826-9a0e-082c7d2ba624 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:28.923857 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 384/742] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:56:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:28.928986 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6f321-f4ac-4df7-988a-fb1d2e77d11a None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:28.928986 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6f321-f4ac-4df7-988a-fb1d2e77d11a None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:28.928986 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6f321-f4ac-4df7-988a-fb1d2e77d11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:28.943679 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-3db6f321-f4ac-4df7-988a-fb1d2e77d11a None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:28.945520 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-3db6f321-f4ac-4df7-988a-fb1d2e77d11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:28.991582 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:29.017928 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 385/743] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:28 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 13:56:29.018776 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db6f321-f4ac-4df7-988a-fb1d2e77d11a None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:29.034532 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc092c-e787-4ab2-8598-b71e359538a3 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:29.073034 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab31d5d-1cf4-499d-bde6-a61b79ea225f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:29.073807 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab31d5d-1cf4-499d-bde6-a61b79ea225f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:29.074091 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab31d5d-1cf4-499d-bde6-a61b79ea225f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:29.074366 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab31d5d-1cf4-499d-bde6-a61b79ea225f None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:29.075062 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab31d5d-1cf4-499d-bde6-a61b79ea225f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:29.075366 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab31d5d-1cf4-499d-bde6-a61b79ea225f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:29.076735 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab31d5d-1cf4-499d-bde6-a61b79ea225f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:29.093443 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 359/744] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:29.093443 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab31d5d-1cf4-499d-bde6-a61b79ea225f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:29.117594 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:29.125845 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db6f321-f4ac-4df7-988a-fb1d2e77d11a None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:29.161225 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad6d2b8-1bb9-4bb7-829f-aa9ccf10b3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:29.162204 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad6d2b8-1bb9-4bb7-829f-aa9ccf10b3ef None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:29.162685 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad6d2b8-1bb9-4bb7-829f-aa9ccf10b3ef None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:29.163709 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad6d2b8-1bb9-4bb7-829f-aa9ccf10b3ef None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:29.164615 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad6d2b8-1bb9-4bb7-829f-aa9ccf10b3ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:29.164922 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad6d2b8-1bb9-4bb7-829f-aa9ccf10b3ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:29.166285 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad6d2b8-1bb9-4bb7-829f-aa9ccf10b3ef None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:29.182738 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 386/745] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:29.182738 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad6d2b8-1bb9-4bb7-829f-aa9ccf10b3ef None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:29.199669 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab31d5d-1cf4-499d-bde6-a61b79ea225f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:29.216499 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:29.237199 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbb3116-3a65-4354-aafa-a9d672d21005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:29.238004 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb3116-3a65-4354-aafa-a9d672d21005 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:29.238289 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb3116-3a65-4354-aafa-a9d672d21005 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:29.238563 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb3116-3a65-4354-aafa-a9d672d21005 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:29.249451 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbb3116-3a65-4354-aafa-a9d672d21005 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:29.251004 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbb3116-3a65-4354-aafa-a9d672d21005 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:29.265695 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-0bbb3116-3a65-4354-aafa-a9d672d21005 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:29.265695 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:29.265695 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:29.265695 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:29.265695 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:29.265695 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:29.265695 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:29.265695 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:29.265695 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:29.265695 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:29.265695 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:29.265695 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:29.265695 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:29.265695 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:29.265695 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:29.266583 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:29.266583 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:29.266583 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:29.266583 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:29.266583 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:29.266583 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:29.266583 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:29.266583 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:29.266583 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:29.266583 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:29.266583 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:29.266583 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:29.266583 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:29.266583 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:29.266583 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:29.266583 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:29.266583 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:29.267450 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:29.267450 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:29.267450 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:29.267450 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:29.267450 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 360/746] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:29.267724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbb3116-3a65-4354-aafa-a9d672d21005 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:29.291930 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad6d2b8-1bb9-4bb7-829f-aa9ccf10b3ef None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:29.332624 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7813c5-2bac-4f97-9eee-b175519ac39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:29.333255 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7813c5-2bac-4f97-9eee-b175519ac39f None admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:29.333542 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7813c5-2bac-4f97-9eee-b175519ac39f None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:29.333819 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7813c5-2bac-4f97-9eee-b175519ac39f None admin] PATH_INFO: `/v3/services` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:29.334508 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7813c5-2bac-4f97-9eee-b175519ac39f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:29.335907 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7813c5-2bac-4f97-9eee-b175519ac39f None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:29.356529 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 387/747] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 13:56:29 2022] POST /identity/v3/services => generated 240 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:56:29.356529 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7813c5-2bac-4f97-9eee-b175519ac39f None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:29.363458 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:29.401518 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbb3116-3a65-4354-aafa-a9d672d21005 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:29.466457 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:29.468269 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ee7a8-42eb-4e4b-8865-414ee732c52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:29.469048 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ee7a8-42eb-4e4b-8865-414ee732c52d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:29.469336 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ee7a8-42eb-4e4b-8865-414ee732c52d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:29.469640 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ee7a8-42eb-4e4b-8865-414ee732c52d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:29.474939 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7813c5-2bac-4f97-9eee-b175519ac39f None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:29.478505 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ee7a8-42eb-4e4b-8865-414ee732c52d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:29.480007 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ee7a8-42eb-4e4b-8865-414ee732c52d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:29.493137 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-ec9ee7a8-42eb-4e4b-8865-414ee732c52d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 13:56:29.493137 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:29.493137 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:29.493137 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:29.493137 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:29.493137 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:29.493137 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:29.493137 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:29.493137 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:29.493137 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:29.493137 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:29.493137 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:29.493137 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:56:29.493137 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:56:29.493137 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:56:29.493137 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:56:29.494318 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:29.494318 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:29.494318 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:56:29.494318 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:29.494318 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:56:29.494318 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:29.494318 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:29.494318 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:29.494318 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:29.494318 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:29.494318 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:29.494318 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:29.494318 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:29.494318 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:29.494318 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:29.494318 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:29.494318 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:29.494318 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:29.495253 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:29.495253 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:29.495253 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:56:29.495253 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:56:29.495253 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:56:29.495253 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:56:29.495253 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:56:29.495253 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:56:29.495253 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 13:56:29.495253 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:29.495253 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 361/748] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 1 13:56:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:29.495253 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ee7a8-42eb-4e4b-8865-414ee732c52d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:29.525670 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1138885f-b1c1-4c54-a459-ed4a642d8406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:29.526414 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138885f-b1c1-4c54-a459-ed4a642d8406 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:29.526703 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138885f-b1c1-4c54-a459-ed4a642d8406 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:29.526979 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138885f-b1c1-4c54-a459-ed4a642d8406 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:29.534014 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1138885f-b1c1-4c54-a459-ed4a642d8406 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:29.535497 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1138885f-b1c1-4c54-a459-ed4a642d8406 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:29.550256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-1138885f-b1c1-4c54-a459-ed4a642d8406 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:29.550256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:29.550256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:29.550256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:29.550256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:29.550256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:29.550256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:29.550256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:29.550256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:29.550256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:29.550256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:29.550256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:29.550256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:56:29.550256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:56:29.550256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:56:29.551158 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:56:29.551158 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:29.551158 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:29.551158 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:29.551158 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:29.551158 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:29.551158 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:29.551158 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:29.551158 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:29.551158 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:29.551158 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:29.551158 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:29.551158 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:29.551158 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:29.551158 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:29.551158 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:29.551158 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:29.552045 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:56:29.552045 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:56:29.552045 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:56:29.552045 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:56:29.552045 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:56:29.552045 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:56:29.552045 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:29.552045 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:29.552045 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 388/749] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:56:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:29.553242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44684790-9329-4902-b1ac-7f6406e6fdfe None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:29.553573 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44684790-9329-4902-b1ac-7f6406e6fdfe None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:29.553856 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44684790-9329-4902-b1ac-7f6406e6fdfe None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:29.555105 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 389/750] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:29.555799 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44684790-9329-4902-b1ac-7f6406e6fdfe None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:29.620669 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ee7a8-42eb-4e4b-8865-414ee732c52d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:29.659653 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3338dc-b1b7-4ca1-84de-6501ef45ff57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:29.660282 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3338dc-b1b7-4ca1-84de-6501ef45ff57 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:29.660594 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3338dc-b1b7-4ca1-84de-6501ef45ff57 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:29.660874 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3338dc-b1b7-4ca1-84de-6501ef45ff57 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:29.661570 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3338dc-b1b7-4ca1-84de-6501ef45ff57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:29.661882 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3338dc-b1b7-4ca1-84de-6501ef45ff57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:29.663901 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3338dc-b1b7-4ca1-84de-6501ef45ff57 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:29.665656 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:29.668932 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44684790-9329-4902-b1ac-7f6406e6fdfe None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:29.677978 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 362/751] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:29.678852 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3338dc-b1b7-4ca1-84de-6501ef45ff57 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:29.712075 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab587b01-675b-43ae-8be7-6e18a7b35c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:29.712723 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab587b01-675b-43ae-8be7-6e18a7b35c4d None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:29.713006 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab587b01-675b-43ae-8be7-6e18a7b35c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:29.713276 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab587b01-675b-43ae-8be7-6e18a7b35c4d None admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:29.714085 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab587b01-675b-43ae-8be7-6e18a7b35c4d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:29.714991 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab587b01-675b-43ae-8be7-6e18a7b35c4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:29.715807 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab587b01-675b-43ae-8be7-6e18a7b35c4d None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:29.737910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 390/752] 10.222.0.48 () {56 vars in 1149 bytes} [Mon Aug 1 13:56:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:29.737910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9459c52-8f5f-471a-97ab-5a0e871d88ef None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:29.737910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9459c52-8f5f-471a-97ab-5a0e871d88ef None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:29.737910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9459c52-8f5f-471a-97ab-5a0e871d88ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:29.754118 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-f9459c52-8f5f-471a-97ab-5a0e871d88ef None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:29.755305 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-f9459c52-8f5f-471a-97ab-5a0e871d88ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:29.766149 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:29.792299 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3338dc-b1b7-4ca1-84de-6501ef45ff57 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:29.831671 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493bf828-53c7-4d08-9c6c-ef914f43e7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:29.836671 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 391/753] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:56:29.836800 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493bf828-53c7-4d08-9c6c-ef914f43e7c1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:29.836800 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493bf828-53c7-4d08-9c6c-ef914f43e7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:29.836800 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493bf828-53c7-4d08-9c6c-ef914f43e7c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:29.837008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9459c52-8f5f-471a-97ab-5a0e871d88ef None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:29.837094 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493bf828-53c7-4d08-9c6c-ef914f43e7c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:29.837094 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493bf828-53c7-4d08-9c6c-ef914f43e7c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:29.838057 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493bf828-53c7-4d08-9c6c-ef914f43e7c1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:29.860876 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 363/754] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 13:56:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:29.861944 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493bf828-53c7-4d08-9c6c-ef914f43e7c1 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:29.920753 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:29.955760 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9459c52-8f5f-471a-97ab-5a0e871d88ef None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:29.981024 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493bf828-53c7-4d08-9c6c-ef914f43e7c1 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:29.993756 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015ce6d-a20e-4384-9dbb-30516eb114e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:29.994305 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015ce6d-a20e-4384-9dbb-30516eb114e5 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:29.994586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015ce6d-a20e-4384-9dbb-30516eb114e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:29.994862 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015ce6d-a20e-4384-9dbb-30516eb114e5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:30.002454 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7015ce6d-a20e-4384-9dbb-30516eb114e5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:30.003901 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7015ce6d-a20e-4384-9dbb-30516eb114e5 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:30.016882 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-7015ce6d-a20e-4384-9dbb-30516eb114e5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 13:56:30.016882 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:30.016882 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:30.016882 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:30.016882 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:30.016882 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:30.016882 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:30.016882 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:30.016882 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:30.016882 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:30.016882 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:30.016882 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:30.016882 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:56:30.016882 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:56:30.016882 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:56:30.016882 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:56:30.017812 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:30.017812 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:30.017812 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:56:30.017812 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:30.017812 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:56:30.017812 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:30.017812 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:30.017812 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:30.017812 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:30.017812 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:30.017812 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:30.017812 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:30.017812 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:30.017812 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:30.017812 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:30.017812 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:30.017812 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:30.017812 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:30.019264 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:30.019264 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:30.019264 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:56:30.019264 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:56:30.019264 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:56:30.019264 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:56:30.019264 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:56:30.019264 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:56:30.019264 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 13:56:30.019264 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:30.019264 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 392/755] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Aug 1 13:56:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:30.021166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded54003-3695-4189-b0ed-7d9221509691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:30.021803 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded54003-3695-4189-b0ed-7d9221509691 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:30.022148 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded54003-3695-4189-b0ed-7d9221509691 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:30.022686 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded54003-3695-4189-b0ed-7d9221509691 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:30.023637 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015ce6d-a20e-4384-9dbb-30516eb114e5 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:30.027382 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:30.101263 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded54003-3695-4189-b0ed-7d9221509691 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:30.102840 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded54003-3695-4189-b0ed-7d9221509691 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:30.133064 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015ce6d-a20e-4384-9dbb-30516eb114e5 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:30.165443 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:30.173108 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f9a23c-6981-4008-ba6a-7618a93f99c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:30.173737 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9a23c-6981-4008-ba6a-7618a93f99c6 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:30.174023 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9a23c-6981-4008-ba6a-7618a93f99c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:30.174303 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9a23c-6981-4008-ba6a-7618a93f99c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:30.175107 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f9a23c-6981-4008-ba6a-7618a93f99c6 None admin] RBAC: Adding query filter params (role.id=58f91e3a13664dabb18c6227a1658fa6, scope.project.id=72ecec9fad964b1fbeaefd0519597229, user.id=2fe2a6d7bd10435ba2c8e552d808b7a0) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:30.175418 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f9a23c-6981-4008-ba6a-7618a93f99c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:30.176863 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f9a23c-6981-4008-ba6a-7618a93f99c6 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:30.192519 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 393/756] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:56:30 2022] GET /identity/v3/role_assignments?user.id=2fe2a6d7bd10435ba2c8e552d808b7a0&scope.project.id=72ecec9fad964b1fbeaefd0519597229&role.id=58f91e3a13664dabb18c6227a1658fa6 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:30.192519 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f9a23c-6981-4008-ba6a-7618a93f99c6 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:30.232670 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 364/757] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:56:29 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 13:56:30.234506 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded54003-3695-4189-b0ed-7d9221509691 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:30.281843 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:30.329685 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f9a23c-6981-4008-ba6a-7618a93f99c6 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:30.359943 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded54003-3695-4189-b0ed-7d9221509691 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:30.377396 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650fd05-8424-4907-899c-a9e53792761c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:30.378057 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650fd05-8424-4907-899c-a9e53792761c None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:30.378356 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650fd05-8424-4907-899c-a9e53792761c None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:30.379622 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650fd05-8424-4907-899c-a9e53792761c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:30.380274 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5650fd05-8424-4907-899c-a9e53792761c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:30.382545 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5650fd05-8424-4907-899c-a9e53792761c None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:30.399398 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-5650fd05-8424-4907-899c-a9e53792761c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 13:56:30.399398 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:30.399398 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:56:30.399398 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:56:30.399398 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:56:30.399398 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:56:30.399398 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:56:30.399398 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:30.399398 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 13:56:30.399398 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:30.399398 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:56:30.399398 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:30.399398 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:30.399398 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:30.399398 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:30.399398 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:30.400353 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:30.400353 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:30.400353 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:30.400353 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:30.400353 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:30.400353 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:30.400353 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:30.400353 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:56:30.400353 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:56:30.400353 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:56:30.400353 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:56:30.400353 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:30.400353 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:30.400353 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:30.400353 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:30.400353 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:30.401238 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:30.401238 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:30.401238 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:30.401238 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:30.401238 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:30.401238 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:30.401238 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:30.401238 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:30.401238 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:30.401238 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:30.401238 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:30.401238 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:56:30.401238 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:30.401238 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 13:56:30.401238 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:30.402599 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 394/758] 10.222.0.48 () {56 vars in 1129 bytes} [Mon Aug 1 13:56:30 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:30.402848 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650fd05-8424-4907-899c-a9e53792761c None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:30.411399 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61987d5a-5d5e-435e-b661-042c00d0678a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:30.412036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61987d5a-5d5e-435e-b661-042c00d0678a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:30.412317 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61987d5a-5d5e-435e-b661-042c00d0678a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:30.412644 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61987d5a-5d5e-435e-b661-042c00d0678a None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:30.414045 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61987d5a-5d5e-435e-b661-042c00d0678a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:30.414356 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61987d5a-5d5e-435e-b661-042c00d0678a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:30.415793 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61987d5a-5d5e-435e-b661-042c00d0678a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:30.440347 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 365/759] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Aug 1 13:56:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:30.440347 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61987d5a-5d5e-435e-b661-042c00d0678a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:30.459008 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:30.542352 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650fd05-8424-4907-899c-a9e53792761c None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:30.578050 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:30.579169 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61987d5a-5d5e-435e-b661-042c00d0678a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:30.587569 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf1a30-0ac4-4f75-8ad8-921f11a0e166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:30.588630 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf1a30-0ac4-4f75-8ad8-921f11a0e166 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:30.589121 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf1a30-0ac4-4f75-8ad8-921f11a0e166 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:30.589604 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf1a30-0ac4-4f75-8ad8-921f11a0e166 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:30.601758 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf1a30-0ac4-4f75-8ad8-921f11a0e166 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:30.603966 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf1a30-0ac4-4f75-8ad8-921f11a0e166 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:30.621121 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969900ab-48bc-4ecf-9389-db97479eba3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:30.621751 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969900ab-48bc-4ecf-9389-db97479eba3e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:30.622034 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969900ab-48bc-4ecf-9389-db97479eba3e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:30.622307 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969900ab-48bc-4ecf-9389-db97479eba3e None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:30.622988 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969900ab-48bc-4ecf-9389-db97479eba3e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:30.623291 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969900ab-48bc-4ecf-9389-db97479eba3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:30.624298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-72cf1a30-0ac4-4f75-8ad8-921f11a0e166 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:30.624298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:30.624298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:30.624298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:30.624298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:30.624298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:30.624298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:30.624298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:30.624298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:30.624298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:30.624298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:30.624298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:30.624298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:30.624298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:30.624298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:30.625333 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:30.625333 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:30.625333 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:30.625333 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:30.625333 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:30.625333 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:30.625333 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:30.625333 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:30.625333 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:30.625333 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:30.625333 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:30.625333 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:30.625333 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:30.625333 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:30.625333 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:30.625333 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:30.625333 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:30.626202 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:30.626202 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:30.626202 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:30.626202 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:30.626413 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969900ab-48bc-4ecf-9389-db97479eba3e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:30.629416 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 395/760] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:30.631143 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf1a30-0ac4-4f75-8ad8-921f11a0e166 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:30.634759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 366/761] 10.222.0.48 () {56 vars in 1124 bytes} [Mon Aug 1 13:56:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:30.635979 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969900ab-48bc-4ecf-9389-db97479eba3e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:30.738518 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:30.757441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf1a30-0ac4-4f75-8ad8-921f11a0e166 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:30.766053 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969900ab-48bc-4ecf-9389-db97479eba3e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:30.794747 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc5466-b5fe-421a-9243-9745025f5197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:30.795728 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc5466-b5fe-421a-9243-9745025f5197 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:30.796218 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc5466-b5fe-421a-9243-9745025f5197 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:30.796727 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc5466-b5fe-421a-9243-9745025f5197 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:30.808410 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464d9ed-1c67-4bab-bfe8-fba372574bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:30.809070 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464d9ed-1c67-4bab-bfe8-fba372574bbf None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:30.809412 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464d9ed-1c67-4bab-bfe8-fba372574bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:30.809613 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464d9ed-1c67-4bab-bfe8-fba372574bbf None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:30.810300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c464d9ed-1c67-4bab-bfe8-fba372574bbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:30.810601 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c464d9ed-1c67-4bab-bfe8-fba372574bbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:30.810966 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fc5466-b5fe-421a-9243-9745025f5197 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:30.811953 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c464d9ed-1c67-4bab-bfe8-fba372574bbf None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:30.812877 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fc5466-b5fe-421a-9243-9745025f5197 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:30.827299 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 367/762] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:30.828143 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464d9ed-1c67-4bab-bfe8-fba372574bbf None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:30.829439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-b8fc5466-b5fe-421a-9243-9745025f5197 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:30.829439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:30.829439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:30.829439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:30.829439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:30.829439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:30.829439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:30.829439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:30.829439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:30.829439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:30.829439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:30.829439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:30.829439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:30.829439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:30.829439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:30.830282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:30.830282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:30.830282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:30.830282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:30.830282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:30.830282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:30.830282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:30.830282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:30.830282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:30.830282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:30.830282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:30.830282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:30.830282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:30.830282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:30.830282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:30.830282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:30.830282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:30.831137 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:30.831137 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:30.831137 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:30.831137 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:30.837613 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 396/763] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:30.838736 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc5466-b5fe-421a-9243-9745025f5197 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:30.876780 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:30.935007 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464d9ed-1c67-4bab-bfe8-fba372574bbf None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:30.949645 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc5466-b5fe-421a-9243-9745025f5197 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:30.970890 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4bbb7-4b46-4709-8bf8-4e75757e3b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:30.971516 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4bbb7-4b46-4709-8bf8-4e75757e3b1d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:30.971796 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4bbb7-4b46-4709-8bf8-4e75757e3b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:30.972071 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4bbb7-4b46-4709-8bf8-4e75757e3b1d None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:30.972778 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae4bbb7-4b46-4709-8bf8-4e75757e3b1d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:30.973080 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae4bbb7-4b46-4709-8bf8-4e75757e3b1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:30.974440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae4bbb7-4b46-4709-8bf8-4e75757e3b1d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:30.977193 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:30.986566 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 368/764] 10.222.0.48 () {56 vars in 1124 bytes} [Mon Aug 1 13:56:30 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:30.987482 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4bbb7-4b46-4709-8bf8-4e75757e3b1d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:30.993001 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98137f-8da4-49b1-96d1-c96fa2842599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:30.994029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98137f-8da4-49b1-96d1-c96fa2842599 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:30.994572 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98137f-8da4-49b1-96d1-c96fa2842599 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:30.995100 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98137f-8da4-49b1-96d1-c96fa2842599 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:31.003327 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e98137f-8da4-49b1-96d1-c96fa2842599 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:31.005213 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e98137f-8da4-49b1-96d1-c96fa2842599 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:31.018982 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-8e98137f-8da4-49b1-96d1-c96fa2842599 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:31.018982 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:31.018982 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:31.018982 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:31.018982 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:31.018982 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:31.018982 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:31.018982 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:31.018982 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:31.018982 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:31.018982 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:31.018982 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:31.018982 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:56:31.018982 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:56:31.018982 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:56:31.020028 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:56:31.020028 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:31.020028 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:31.020028 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:31.020028 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:31.020028 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:31.020028 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:31.020028 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:31.020028 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:31.020028 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:31.020028 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:31.020028 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:31.020028 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:31.020028 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:31.020028 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:31.020028 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:31.020028 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:31.021096 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:56:31.021096 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:56:31.021096 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:56:31.021096 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:56:31.021096 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:56:31.021096 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:56:31.021096 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:31.021096 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:31.024287 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 397/765] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:56:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:31.025446 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98137f-8da4-49b1-96d1-c96fa2842599 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:31.094813 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4bbb7-4b46-4709-8bf8-4e75757e3b1d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:31.127858 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:31.131020 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3611e-d99e-4718-b796-85201ad40b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:31.131839 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3611e-d99e-4718-b796-85201ad40b98 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:31.132075 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3611e-d99e-4718-b796-85201ad40b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:31.132348 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3611e-d99e-4718-b796-85201ad40b98 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:31.133083 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de3611e-d99e-4718-b796-85201ad40b98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:31.133447 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de3611e-d99e-4718-b796-85201ad40b98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:31.134739 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de3611e-d99e-4718-b796-85201ad40b98 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:31.137874 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98137f-8da4-49b1-96d1-c96fa2842599 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:31.152149 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 369/766] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:31.152149 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3611e-d99e-4718-b796-85201ad40b98 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:31.179914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb41e5f5-00cf-4e01-ab16-91da0d445a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:31.180937 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41e5f5-00cf-4e01-ab16-91da0d445a03 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:31.181431 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41e5f5-00cf-4e01-ab16-91da0d445a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:31.181915 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41e5f5-00cf-4e01-ab16-91da0d445a03 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:31.183034 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb41e5f5-00cf-4e01-ab16-91da0d445a03 None admin] RBAC: Adding query filter params (interface=public, service_id=b6c53d9f08764b6b844bd62ae9d13c71, region_id=RegionOne) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:31.183568 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb41e5f5-00cf-4e01-ab16-91da0d445a03 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:31.185183 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb41e5f5-00cf-4e01-ab16-91da0d445a03 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:31.193700 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 398/767] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Aug 1 13:56:31 2022] GET /identity/v3/endpoints?service_id=b6c53d9f08764b6b844bd62ae9d13c71&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:31.194804 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb41e5f5-00cf-4e01-ab16-91da0d445a03 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:31.219811 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:31.270658 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3611e-d99e-4718-b796-85201ad40b98 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:31.307832 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb41e5f5-00cf-4e01-ab16-91da0d445a03 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:31.317569 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb97259-98dd-41ae-9896-52c553c106ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:31.318777 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb97259-98dd-41ae-9896-52c553c106ee None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:31.319159 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb97259-98dd-41ae-9896-52c553c106ee None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:31.319531 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb97259-98dd-41ae-9896-52c553c106ee None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:31.320750 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb97259-98dd-41ae-9896-52c553c106ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:31.321348 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb97259-98dd-41ae-9896-52c553c106ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:31.323150 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb97259-98dd-41ae-9896-52c553c106ee None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:31.341478 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 370/768] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 13:56:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:31.342737 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb97259-98dd-41ae-9896-52c553c106ee None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:31.346582 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ebc0ea-3ca4-4df9-a4eb-ba42e3ad5dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:31.347259 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebc0ea-3ca4-4df9-a4eb-ba42e3ad5dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:31.347541 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebc0ea-3ca4-4df9-a4eb-ba42e3ad5dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:31.347813 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebc0ea-3ca4-4df9-a4eb-ba42e3ad5dd6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:31.355942 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ebc0ea-3ca4-4df9-a4eb-ba42e3ad5dd6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:31.357444 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ebc0ea-3ca4-4df9-a4eb-ba42e3ad5dd6 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:31.373536 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-42ebc0ea-3ca4-4df9-a4eb-ba42e3ad5dd6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:31.373536 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:31.373536 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:31.373536 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:31.373536 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:31.373536 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:31.373536 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:31.373536 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:31.373536 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:31.373536 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:31.373536 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:31.373536 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:31.373536 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:56:31.373536 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:56:31.373536 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:56:31.374455 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:56:31.374455 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:31.374455 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:31.374455 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:31.374455 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:31.374455 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:31.374455 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:31.374455 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:31.374455 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:31.374455 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:31.374455 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:31.374455 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:31.374455 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:31.374455 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:31.374455 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:31.374455 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:31.374455 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:31.375339 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:56:31.375339 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:56:31.375339 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:56:31.375339 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:56:31.375339 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:56:31.375339 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:56:31.375339 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:31.375339 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:31.375761 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 399/769] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:56:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:31.377393 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ebc0ea-3ca4-4df9-a4eb-ba42e3ad5dd6 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:31.406774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:31.463532 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb97259-98dd-41ae-9896-52c553c106ee None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:31.486921 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:31.501773 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f213d0-9bc3-4f1b-80ad-a1562222ac6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:31.502395 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f213d0-9bc3-4f1b-80ad-a1562222ac6a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:31.502681 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f213d0-9bc3-4f1b-80ad-a1562222ac6a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:31.502955 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f213d0-9bc3-4f1b-80ad-a1562222ac6a None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:31.503494 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ebc0ea-3ca4-4df9-a4eb-ba42e3ad5dd6 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:31.503619 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f213d0-9bc3-4f1b-80ad-a1562222ac6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:31.504981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f213d0-9bc3-4f1b-80ad-a1562222ac6a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:31.511165 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 371/770] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Aug 1 13:56:31 2022] GET /identity/v3/services => generated 772 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:31.511931 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f213d0-9bc3-4f1b-80ad-a1562222ac6a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:31.552790 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c3ff8e-2be5-4572-b296-100c31aca41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:31.554082 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3ff8e-2be5-4572-b296-100c31aca41e None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:31.554525 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3ff8e-2be5-4572-b296-100c31aca41e None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:31.554951 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3ff8e-2be5-4572-b296-100c31aca41e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:31.555926 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c3ff8e-2be5-4572-b296-100c31aca41e None admin] RBAC: Adding query filter params (role.id=d3d8169e71454005a8cf0d7544ad05dc, scope.project.id=72ecec9fad964b1fbeaefd0519597229, user.id=5007a2f8e6244b0788f0a554cc572fe5) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:31.556529 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c3ff8e-2be5-4572-b296-100c31aca41e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:31.558270 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c3ff8e-2be5-4572-b296-100c31aca41e None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:31.565049 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 400/771] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:56:31 2022] GET /identity/v3/role_assignments?user.id=5007a2f8e6244b0788f0a554cc572fe5&scope.project.id=72ecec9fad964b1fbeaefd0519597229&role.id=d3d8169e71454005a8cf0d7544ad05dc => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:31.573769 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed6d90-3a31-48e3-9da4-801d63bed35b None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:31.574342 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed6d90-3a31-48e3-9da4-801d63bed35b None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:31.574629 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed6d90-3a31-48e3-9da4-801d63bed35b None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:31.576315 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 401/772] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:31.591271 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e2b4b-1d14-4d62-b16d-62ed8de68872 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:31.591592 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e2b4b-1d14-4d62-b16d-62ed8de68872 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:31.591843 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e2b4b-1d14-4d62-b16d-62ed8de68872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:31.629653 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-fe7e2b4b-1d14-4d62-b16d-62ed8de68872 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:31.636782 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-fe7e2b4b-1d14-4d62-b16d-62ed8de68872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:31.679007 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f213d0-9bc3-4f1b-80ad-a1562222ac6a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:31.733670 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c346a7-457f-472d-be5f-38dfc6f85892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:31.734320 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c346a7-457f-472d-be5f-38dfc6f85892 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:31.734605 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c346a7-457f-472d-be5f-38dfc6f85892 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:31.734879 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c346a7-457f-472d-be5f-38dfc6f85892 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:31.735650 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c346a7-457f-472d-be5f-38dfc6f85892 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:31.735963 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c346a7-457f-472d-be5f-38dfc6f85892 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:31.737460 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c346a7-457f-472d-be5f-38dfc6f85892 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:31.745506 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 402/773] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:56:31.752449 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 372/774] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 13:56:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:31.759307 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e2b4b-1d14-4d62-b16d-62ed8de68872 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:31.772938 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c346a7-457f-472d-be5f-38dfc6f85892 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:31.776940 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:31.847642 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:31.925992 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e2b4b-1d14-4d62-b16d-62ed8de68872 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:31.946961 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c346a7-457f-472d-be5f-38dfc6f85892 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:31.993504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d937c-d01d-4109-ba24-99d89a69c9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:31.994590 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d937c-d01d-4109-ba24-99d89a69c9c7 None admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:31.995099 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d937c-d01d-4109-ba24-99d89a69c9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:31.995591 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d937c-d01d-4109-ba24-99d89a69c9c7 None admin] PATH_INFO: `/v3/projects/72ecec9fad964b1fbeaefd0519597229/users/a246654c783b4cc68d212e91bb05d8c6/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:31.999126 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d937c-d01d-4109-ba24-99d89a69c9c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=72ecec9fad964b1fbeaefd0519597229, user_id=a246654c783b4cc68d212e91bb05d8c6, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:32.001005 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d937c-d01d-4109-ba24-99d89a69c9c7 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:32.012040 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a22df-839a-45c9-8c06-37581d761fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:32.013162 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a22df-839a-45c9-8c06-37581d761fd0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:32.013670 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a22df-839a-45c9-8c06-37581d761fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:32.014146 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a22df-839a-45c9-8c06-37581d761fd0 None admin] PATH_INFO: `/v3/services/network` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:32.015023 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52a22df-839a-45c9-8c06-37581d761fd0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:32.016701 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52a22df-839a-45c9-8c06-37581d761fd0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:32.029234 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 403/775] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 13:56:31 2022] PUT /identity/v3/projects/72ecec9fad964b1fbeaefd0519597229/users/a246654c783b4cc68d212e91bb05d8c6/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:56:32.035724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-d52a22df-839a-45c9-8c06-37581d761fd0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 13:56:32.035724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:32.035724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:56:32.035724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:56:32.035724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:56:32.035724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:56:32.035724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:56:32.035724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:32.035724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 13:56:32.035724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:32.035724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:56:32.035724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:32.035724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:32.035724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:32.035724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:32.035724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:32.035724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:32.036785 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:32.036785 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:32.036785 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:32.036785 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:32.036785 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:32.036785 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:32.036785 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:56:32.036785 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:56:32.036785 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:56:32.036785 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:56:32.036785 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:32.036785 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:32.036785 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:32.036785 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:32.036785 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:32.036785 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:32.037664 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:32.037664 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:32.037664 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:32.037664 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:32.037664 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:32.037664 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:32.037664 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:32.037664 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:32.037664 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:32.037664 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:32.037664 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:56:32.037664 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:32.037664 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 13:56:32.037664 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:32.040642 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 373/776] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:56:31 2022] GET /identity/v3/services/network => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:56:32.049612 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d937c-d01d-4109-ba24-99d89a69c9c7 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:32.148470 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:32.181528 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d937c-d01d-4109-ba24-99d89a69c9c7 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:32.237423 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5808528a-9e4c-4b24-b9b1-27b7fa8435e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:32.238065 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808528a-9e4c-4b24-b9b1-27b7fa8435e2 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:32.238345 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808528a-9e4c-4b24-b9b1-27b7fa8435e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:32.238621 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808528a-9e4c-4b24-b9b1-27b7fa8435e2 None admin] PATH_INFO: `/v3/services` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:32.239322 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5808528a-9e4c-4b24-b9b1-27b7fa8435e2 None admin] RBAC: Adding query filter params (name=network) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:32.239626 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5808528a-9e4c-4b24-b9b1-27b7fa8435e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:32.241702 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5808528a-9e4c-4b24-b9b1-27b7fa8435e2 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:32.251266 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 404/777] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 13:56:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:32.257435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a22df-839a-45c9-8c06-37581d761fd0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:32.309900 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c0ba69-7579-47fa-9942-688715ef04d3 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:32.310208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c0ba69-7579-47fa-9942-688715ef04d3 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:32.310477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c0ba69-7579-47fa-9942-688715ef04d3 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:32.313146 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 405/778] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:56:32.318245 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02adb39a-43eb-429c-934c-6782808f372b None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:32.318536 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02adb39a-43eb-429c-934c-6782808f372b None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:32.318846 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02adb39a-43eb-429c-934c-6782808f372b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:32.343052 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-02adb39a-43eb-429c-934c-6782808f372b None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:32.344727 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-02adb39a-43eb-429c-934c-6782808f372b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:32.410396 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a22df-839a-45c9-8c06-37581d761fd0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:32.426981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:32.428941 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 406/779] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:56:32.454580 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02adb39a-43eb-429c-934c-6782808f372b None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:32.466557 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d11d7d-04df-42cf-be67-f3c403aabebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:32.467187 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d11d7d-04df-42cf-be67-f3c403aabebe None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:32.467469 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d11d7d-04df-42cf-be67-f3c403aabebe None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:32.467741 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d11d7d-04df-42cf-be67-f3c403aabebe None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:32.468430 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d11d7d-04df-42cf-be67-f3c403aabebe None admin] RBAC: Adding query filter params (type=network) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:32.468760 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d11d7d-04df-42cf-be67-f3c403aabebe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:32.470120 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d11d7d-04df-42cf-be67-f3c403aabebe None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:32.481053 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 374/780] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 13:56:32 2022] GET /identity/v3/services?type=network => generated 350 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:32.499156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d11d7d-04df-42cf-be67-f3c403aabebe None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:32.534329 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:32.574443 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02adb39a-43eb-429c-934c-6782808f372b None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:32.614824 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39268e66-8994-42dd-904b-de1d7fb7a721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:32.615795 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39268e66-8994-42dd-904b-de1d7fb7a721 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:32.616223 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39268e66-8994-42dd-904b-de1d7fb7a721 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:32.616668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39268e66-8994-42dd-904b-de1d7fb7a721 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:32.628106 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39268e66-8994-42dd-904b-de1d7fb7a721 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:56:32.634193 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-39268e66-8994-42dd-904b-de1d7fb7a721 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:56:32.634193 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:32.634193 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:32.634193 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:32.634193 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:32.634193 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:32.634193 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:32.634193 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:32.634193 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:32.634193 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:32.634193 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:32.634193 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:32.634193 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:56:32.634193 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:56:32.634193 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:56:32.634193 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:56:32.635256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:56:32.635256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:56:32.635256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:56:32.635256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:56:32.635256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:32.635256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:32.635256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:32.635256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:32.635256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:32.635256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:32.635256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:32.635256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:32.635256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:32.635256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:32.635256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:32.635256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:32.636242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:32.636242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:32.636242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:32.636242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:32.636242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:56:32.636242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:56:32.636242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:56:32.636242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:56:32.636242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:56:32.636242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:56:32.636242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:56:32.636242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:32.638612 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 407/781] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 13:56:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:32.644127 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39268e66-8994-42dd-904b-de1d7fb7a721 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:32.741030 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d11d7d-04df-42cf-be67-f3c403aabebe None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:32.773792 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39268e66-8994-42dd-904b-de1d7fb7a721 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:32.805506 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c4ebf-5da7-4022-bed4-b44aa1361411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:32.807397 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c4ebf-5da7-4022-bed4-b44aa1361411 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:32.809244 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c4ebf-5da7-4022-bed4-b44aa1361411 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:32.809728 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c4ebf-5da7-4022-bed4-b44aa1361411 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:32.810694 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c4ebf-5da7-4022-bed4-b44aa1361411 None admin] RBAC: Adding query filter params (interface=public, service_id=d645281cbe834034b60fbceaa966c65e, region_id=RegionOne) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:32.812155 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c4ebf-5da7-4022-bed4-b44aa1361411 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:32.815512 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c4ebf-5da7-4022-bed4-b44aa1361411 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:32.831626 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 375/782] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Aug 1 13:56:32 2022] GET /identity/v3/endpoints?service_id=d645281cbe834034b60fbceaa966c65e&interface=public®ion_id=RegionOne => generated 194 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:32.836650 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:32.845856 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c4ebf-5da7-4022-bed4-b44aa1361411 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:32.852207 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea13810-f7c0-4dca-9738-7274ecf96ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:32.852883 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea13810-f7c0-4dca-9738-7274ecf96ddf None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:32.853170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea13810-f7c0-4dca-9738-7274ecf96ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:32.853448 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea13810-f7c0-4dca-9738-7274ecf96ddf None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:32.854190 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea13810-f7c0-4dca-9738-7274ecf96ddf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:32.854500 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea13810-f7c0-4dca-9738-7274ecf96ddf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:32.855890 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea13810-f7c0-4dca-9738-7274ecf96ddf None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:32.865964 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 408/783] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:56:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:32.871709 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea13810-f7c0-4dca-9738-7274ecf96ddf None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:32.909304 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:33.026139 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c4ebf-5da7-4022-bed4-b44aa1361411 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:33.058513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea13810-f7c0-4dca-9738-7274ecf96ddf None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:33.090112 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e006f187-c3e7-4365-978e-ab007de1a216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:33.090837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006f187-c3e7-4365-978e-ab007de1a216 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:33.091132 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006f187-c3e7-4365-978e-ab007de1a216 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:33.091412 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006f187-c3e7-4365-978e-ab007de1a216 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:33.092141 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e006f187-c3e7-4365-978e-ab007de1a216 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:33.093552 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e006f187-c3e7-4365-978e-ab007de1a216 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:33.102435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 376/784] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Aug 1 13:56:32 2022] GET /identity/v3/services => generated 772 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:33.129399 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57a35a-3615-43e4-9c70-3a0c2f9a652f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:33.130091 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57a35a-3615-43e4-9c70-3a0c2f9a652f None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:33.130622 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57a35a-3615-43e4-9c70-3a0c2f9a652f None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:33.130947 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57a35a-3615-43e4-9c70-3a0c2f9a652f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:33.143909 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af57a35a-3615-43e4-9c70-3a0c2f9a652f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:33.145400 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af57a35a-3615-43e4-9c70-3a0c2f9a652f None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:33.161397 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-af57a35a-3615-43e4-9c70-3a0c2f9a652f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:33.161397 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:33.161397 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:33.161397 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:33.161397 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:33.161397 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:33.161397 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:33.161397 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:33.161397 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:33.161397 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:33.161397 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:33.161397 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:33.161397 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:33.161397 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:33.161397 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:33.162340 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:33.162340 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:33.162340 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:33.162340 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:33.162340 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:33.162340 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:33.162340 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:33.162340 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:33.162340 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:33.162340 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:33.162340 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:33.162340 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:33.162340 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:33.162340 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:33.162340 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:33.162340 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:33.162340 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:33.163298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:33.163298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:33.163298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:33.163298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:33.167289 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 409/785] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:33.168750 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e006f187-c3e7-4365-978e-ab007de1a216 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:33.302907 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e006f187-c3e7-4365-978e-ab007de1a216 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:33.307255 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:33.340823 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665e838-de37-4be8-bc90-b92ccdfcf13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:33.341450 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665e838-de37-4be8-bc90-b92ccdfcf13b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:33.341729 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665e838-de37-4be8-bc90-b92ccdfcf13b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:33.342004 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665e838-de37-4be8-bc90-b92ccdfcf13b None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:33.342701 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c665e838-de37-4be8-bc90-b92ccdfcf13b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:33.343004 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c665e838-de37-4be8-bc90-b92ccdfcf13b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:33.344363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c665e838-de37-4be8-bc90-b92ccdfcf13b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:33.357025 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 377/786] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:33.362351 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57a35a-3615-43e4-9c70-3a0c2f9a652f None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:33.421380 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:33.501904 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57a35a-3615-43e4-9c70-3a0c2f9a652f None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:33.550566 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed6cbb0-b162-473b-a6b8-0186d062d38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:33.551223 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6cbb0-b162-473b-a6b8-0186d062d38d None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:33.551507 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6cbb0-b162-473b-a6b8-0186d062d38d None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:33.551787 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6cbb0-b162-473b-a6b8-0186d062d38d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:33.565677 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed6cbb0-b162-473b-a6b8-0186d062d38d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:33.567063 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed6cbb0-b162-473b-a6b8-0186d062d38d None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:33.582029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-3ed6cbb0-b162-473b-a6b8-0186d062d38d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 13:56:33.582029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:33.582029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:33.582029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:33.582029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:33.582029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:33.582029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:33.582029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:33.582029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:33.582029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:33.582029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:33.582029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:33.582029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:56:33.582029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:56:33.582029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:56:33.582029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:56:33.583378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:33.583378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:33.583378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:56:33.583378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:33.583378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:56:33.583378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:33.583378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:33.583378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:33.583378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:33.583378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:33.583378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:33.583378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:33.583378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:33.583378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:33.583378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:33.583378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:33.583378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:33.583378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:33.584320 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:33.584320 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:33.584320 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:56:33.584320 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:56:33.584320 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:56:33.584320 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:56:33.584320 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:56:33.584320 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:56:33.584320 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 13:56:33.584320 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:33.584320 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 410/787] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 13:56:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:33.589013 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665e838-de37-4be8-bc90-b92ccdfcf13b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:33.732923 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665e838-de37-4be8-bc90-b92ccdfcf13b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:33.782116 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a0c181-dc12-4496-956e-99491ec65a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:33.783548 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0c181-dc12-4496-956e-99491ec65a77 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:33.784027 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0c181-dc12-4496-956e-99491ec65a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:33.784528 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0c181-dc12-4496-956e-99491ec65a77 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:33.785578 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a0c181-dc12-4496-956e-99491ec65a77 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:33.786083 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a0c181-dc12-4496-956e-99491ec65a77 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:33.788240 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a0c181-dc12-4496-956e-99491ec65a77 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:33.814363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 378/788] 10.222.0.48 () {56 vars in 1135 bytes} [Mon Aug 1 13:56:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:33.821577 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed6cbb0-b162-473b-a6b8-0186d062d38d None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:33.977421 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed6cbb0-b162-473b-a6b8-0186d062d38d None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:33.978947 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:34.026496 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c7a44-dc94-4930-b37a-94258056ab15 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:34.027119 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c7a44-dc94-4930-b37a-94258056ab15 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:34.027597 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c7a44-dc94-4930-b37a-94258056ab15 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:34.029491 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 379/789] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:34.034858 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e65e56-1504-4b1a-9263-debb3908eab1 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:34.035144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e65e56-1504-4b1a-9263-debb3908eab1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:34.035419 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e65e56-1504-4b1a-9263-debb3908eab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:34.039484 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baeaa68-4959-4a2c-8f4b-c668795f7d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:34.040144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baeaa68-4959-4a2c-8f4b-c668795f7d3c None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:34.040424 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baeaa68-4959-4a2c-8f4b-c668795f7d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:34.040723 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baeaa68-4959-4a2c-8f4b-c668795f7d3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:34.062276 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baeaa68-4959-4a2c-8f4b-c668795f7d3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:34.063750 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baeaa68-4959-4a2c-8f4b-c668795f7d3c None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:34.078485 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-27e65e56-1504-4b1a-9263-debb3908eab1 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:34.079611 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-27e65e56-1504-4b1a-9263-debb3908eab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:34.097668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-1baeaa68-4959-4a2c-8f4b-c668795f7d3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:34.097668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:34.097668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:34.097668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:34.097668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:34.097668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:34.097668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:34.097668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:34.097668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:34.097668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:34.097668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:34.097668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:34.097668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:34.097668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:34.097668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:34.098576 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:34.098576 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:34.098576 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:34.098576 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:34.098576 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:34.098576 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:34.098576 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:34.098576 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:34.098576 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:34.098576 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:34.098576 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:34.098576 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:34.098576 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:34.098576 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:34.098576 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:34.098576 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:34.098576 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:34.099433 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:34.099433 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:34.099433 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:34.099433 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:34.106003 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 411/790] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:34.108642 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baeaa68-4959-4a2c-8f4b-c668795f7d3c None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:34.116907 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:34.180570 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 380/791] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:56:34.182255 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02370e-8f01-4874-a5aa-f61262ec18f6 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:34.182745 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02370e-8f01-4874-a5aa-f61262ec18f6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:34.183218 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02370e-8f01-4874-a5aa-f61262ec18f6 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:34.184994 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 381/792] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:34.195949 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff219f5-2014-4943-a3a8-5a4662300cbd None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:34.196520 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff219f5-2014-4943-a3a8-5a4662300cbd None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:34.197015 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff219f5-2014-4943-a3a8-5a4662300cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:34.215023 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baeaa68-4959-4a2c-8f4b-c668795f7d3c None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:34.226434 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-fff219f5-2014-4943-a3a8-5a4662300cbd None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:34.228043 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fff219f5-2014-4943-a3a8-5a4662300cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:34.250093 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1862d6b1-f6b2-4a74-90c2-91d470f591e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:34.250481 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862d6b1-f6b2-4a74-90c2-91d470f591e7 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:34.250764 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862d6b1-f6b2-4a74-90c2-91d470f591e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:34.251038 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862d6b1-f6b2-4a74-90c2-91d470f591e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:34.251739 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1862d6b1-f6b2-4a74-90c2-91d470f591e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:34.252042 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1862d6b1-f6b2-4a74-90c2-91d470f591e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:34.253435 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1862d6b1-f6b2-4a74-90c2-91d470f591e7 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:34.270032 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 412/793] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:34.270902 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1862d6b1-f6b2-4a74-90c2-91d470f591e7 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:34.296910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:34.308716 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 382/794] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Aug 01 13:56:34.309831 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff219f5-2014-4943-a3a8-5a4662300cbd None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:34.374873 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1862d6b1-f6b2-4a74-90c2-91d470f591e7 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:34.411145 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0de36d-08d0-40da-ae49-8f6da440c76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:34.411384 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:34.411792 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0de36d-08d0-40da-ae49-8f6da440c76a None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:34.412074 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0de36d-08d0-40da-ae49-8f6da440c76a None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:34.412392 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0de36d-08d0-40da-ae49-8f6da440c76a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:34.413077 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0de36d-08d0-40da-ae49-8f6da440c76a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:34.414505 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0de36d-08d0-40da-ae49-8f6da440c76a None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:34.423587 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-fb0de36d-08d0-40da-ae49-8f6da440c76a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 13:56:34.423587 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:34.423587 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:56:34.423587 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:56:34.423587 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:56:34.423587 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:56:34.423587 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:56:34.423587 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:34.423587 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 13:56:34.423587 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:34.423587 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:56:34.423587 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:34.423587 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:34.423587 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:34.423587 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:34.423587 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:34.424581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:34.424581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:34.424581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:34.424581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:34.424581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:34.424581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:34.424581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:34.424581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:56:34.424581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:56:34.424581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:56:34.424581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:56:34.424581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:34.424581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:34.424581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:34.424581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:34.424581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:34.429357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:34.429357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:34.429357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:34.429357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:34.429357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:34.429357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:34.429357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:34.429357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:34.429357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:34.429357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:34.429357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:34.429357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:56:34.429357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:34.429357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 13:56:34.429357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:34.429357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 413/795] 10.222.0.48 () {56 vars in 1129 bytes} [Mon Aug 1 13:56:34 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:34.430389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0de36d-08d0-40da-ae49-8f6da440c76a None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:34.446022 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff219f5-2014-4943-a3a8-5a4662300cbd None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:34.489125 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19411f1e-cf03-4740-993e-f5b95a1eb2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:34.489782 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19411f1e-cf03-4740-993e-f5b95a1eb2a2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:34.490071 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19411f1e-cf03-4740-993e-f5b95a1eb2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:34.490353 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19411f1e-cf03-4740-993e-f5b95a1eb2a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:34.497473 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19411f1e-cf03-4740-993e-f5b95a1eb2a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:34.498910 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19411f1e-cf03-4740-993e-f5b95a1eb2a2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:34.511144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-19411f1e-cf03-4740-993e-f5b95a1eb2a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:34.511144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:34.511144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:34.511144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:34.511144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:34.511144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:34.511144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:34.511144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:34.511144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:34.511144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:34.511144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:34.511144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:34.511144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:56:34.511144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:56:34.511144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:56:34.512046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:56:34.512046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:34.512046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:34.512046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:34.512046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:34.512046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:34.512046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:34.512046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:34.512046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:34.512046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:34.512046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:34.512046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:34.512046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:34.512046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:34.512046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:34.512046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:34.512046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:34.512983 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:56:34.512983 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:56:34.512983 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:56:34.512983 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:56:34.512983 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:56:34.512983 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:56:34.512983 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:34.512983 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:34.512983 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 383/796] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:56:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:34.513457 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19411f1e-cf03-4740-993e-f5b95a1eb2a2 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:34.529558 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:34.553172 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0de36d-08d0-40da-ae49-8f6da440c76a None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:34.589566 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab13306-79b0-441f-820f-628e0963cb7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:34.590227 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab13306-79b0-441f-820f-628e0963cb7e None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:34.590905 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab13306-79b0-441f-820f-628e0963cb7e None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:34.590905 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab13306-79b0-441f-820f-628e0963cb7e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:34.603998 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab13306-79b0-441f-820f-628e0963cb7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:56:34.610682 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-fab13306-79b0-441f-820f-628e0963cb7e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:56:34.610682 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:34.610682 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:34.610682 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:34.610682 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:34.610682 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:34.610682 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:34.610682 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:34.610682 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:34.610682 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:34.610682 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:34.610682 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:34.610682 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:56:34.610682 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:56:34.610682 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:56:34.610682 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:56:34.611600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:56:34.611600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:56:34.611600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:56:34.611600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:56:34.611600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:34.611600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:34.611600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:34.611600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:34.611600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:34.611600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:34.611600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:34.611600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:34.611600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:34.611600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:34.611600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:34.611600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:34.612441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:34.612441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:34.612441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:34.612441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:34.612441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:56:34.612441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:56:34.612441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:56:34.612441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:56:34.612441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:56:34.612441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:56:34.612441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:56:34.612441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:34.613104 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 414/797] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 13:56:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:34.613736 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a8529-7d02-4da0-9475-03ec8423bce3 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:34.614019 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a8529-7d02-4da0-9475-03ec8423bce3 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:34.614457 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a8529-7d02-4da0-9475-03ec8423bce3 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:34.619876 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 415/798] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:34.619876 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a8529-7d02-4da0-9475-03ec8423bce3 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:34.628976 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19411f1e-cf03-4740-993e-f5b95a1eb2a2 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:34.666185 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f3a4ff-cf24-4adf-bc0b-0d8d231ff8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:34.666805 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f3a4ff-cf24-4adf-bc0b-0d8d231ff8df None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:34.667087 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f3a4ff-cf24-4adf-bc0b-0d8d231ff8df None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:34.667361 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f3a4ff-cf24-4adf-bc0b-0d8d231ff8df None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:34.668046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f3a4ff-cf24-4adf-bc0b-0d8d231ff8df None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:34.668358 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f3a4ff-cf24-4adf-bc0b-0d8d231ff8df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:34.669744 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f3a4ff-cf24-4adf-bc0b-0d8d231ff8df None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:34.674567 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 384/799] 10.222.0.48 () {56 vars in 1124 bytes} [Mon Aug 1 13:56:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:34.679640 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f3a4ff-cf24-4adf-bc0b-0d8d231ff8df None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:34.682729 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:34.735605 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a8529-7d02-4da0-9475-03ec8423bce3 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:34.775439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b5c23f-48b8-41e6-b442-20c0826bcec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:34.775628 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b5c23f-48b8-41e6-b442-20c0826bcec9 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:34.777215 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b5c23f-48b8-41e6-b442-20c0826bcec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:34.777215 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b5c23f-48b8-41e6-b442-20c0826bcec9 None admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:34.777215 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b5c23f-48b8-41e6-b442-20c0826bcec9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:34.777648 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b5c23f-48b8-41e6-b442-20c0826bcec9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:34.779617 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b5c23f-48b8-41e6-b442-20c0826bcec9 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:34.787102 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:34.798416 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 416/800] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 13:56:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:34.798416 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414ddc3-d575-4b55-9b83-6386161249da None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:34.798709 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f3a4ff-cf24-4adf-bc0b-0d8d231ff8df None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:34.799124 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414ddc3-d575-4b55-9b83-6386161249da None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:34.799124 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414ddc3-d575-4b55-9b83-6386161249da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:34.816065 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-3414ddc3-d575-4b55-9b83-6386161249da None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:34.817438 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-3414ddc3-d575-4b55-9b83-6386161249da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:34.839777 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c0405-870c-438b-83ae-27fec9d648f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:34.840407 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c0405-870c-438b-83ae-27fec9d648f5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:34.840717 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c0405-870c-438b-83ae-27fec9d648f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:34.840997 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c0405-870c-438b-83ae-27fec9d648f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:34.841714 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5c0405-870c-438b-83ae-27fec9d648f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:34.842022 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5c0405-870c-438b-83ae-27fec9d648f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:34.843386 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5c0405-870c-438b-83ae-27fec9d648f5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:34.851490 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 385/801] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 13:56:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:34.852274 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c0405-870c-438b-83ae-27fec9d648f5 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:34.894291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 417/802] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:56:34.895036 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414ddc3-d575-4b55-9b83-6386161249da None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:34.931305 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:34.973021 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c0405-870c-438b-83ae-27fec9d648f5 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:35.005761 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414ddc3-d575-4b55-9b83-6386161249da None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:35.009963 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e898ad8-cbf3-46f3-b0eb-e7ad960c7d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:35.010591 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e898ad8-cbf3-46f3-b0eb-e7ad960c7d98 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:35.010879 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e898ad8-cbf3-46f3-b0eb-e7ad960c7d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:35.011159 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e898ad8-cbf3-46f3-b0eb-e7ad960c7d98 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:35.011847 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e898ad8-cbf3-46f3-b0eb-e7ad960c7d98 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:35.012158 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e898ad8-cbf3-46f3-b0eb-e7ad960c7d98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:35.013563 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e898ad8-cbf3-46f3-b0eb-e7ad960c7d98 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:35.023519 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 386/803] 10.222.0.48 () {56 vars in 1124 bytes} [Mon Aug 1 13:56:34 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:35.023519 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e898ad8-cbf3-46f3-b0eb-e7ad960c7d98 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:35.041143 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:35.049625 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e90fb3-1c6c-4991-ab0b-430afa1b179e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:35.050305 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e90fb3-1c6c-4991-ab0b-430afa1b179e None admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:35.050590 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e90fb3-1c6c-4991-ab0b-430afa1b179e None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:35.050868 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e90fb3-1c6c-4991-ab0b-430afa1b179e None admin] PATH_INFO: `/v3/projects/72ecec9fad964b1fbeaefd0519597229/users/2fe2a6d7bd10435ba2c8e552d808b7a0/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:35.053354 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e90fb3-1c6c-4991-ab0b-430afa1b179e None admin] RBAC: Authorizing `identity:create_grant(project_id=72ecec9fad964b1fbeaefd0519597229, user_id=2fe2a6d7bd10435ba2c8e552d808b7a0, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:35.054796 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e90fb3-1c6c-4991-ab0b-430afa1b179e None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:35.070067 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 418/804] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 13:56:34 2022] PUT /identity/v3/projects/72ecec9fad964b1fbeaefd0519597229/users/2fe2a6d7bd10435ba2c8e552d808b7a0/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:56:35.070985 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e90fb3-1c6c-4991-ab0b-430afa1b179e None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:35.163430 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e898ad8-cbf3-46f3-b0eb-e7ad960c7d98 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:35.198196 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e90fb3-1c6c-4991-ab0b-430afa1b179e None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:35.201130 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:35.211733 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f9219-42af-4a12-aaf4-bc345d0e5e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:35.212545 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f9219-42af-4a12-aaf4-bc345d0e5e19 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:35.212906 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f9219-42af-4a12-aaf4-bc345d0e5e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:35.213318 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f9219-42af-4a12-aaf4-bc345d0e5e19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:35.228040 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90f9219-42af-4a12-aaf4-bc345d0e5e19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:35.229534 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90f9219-42af-4a12-aaf4-bc345d0e5e19 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:35.245339 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1396be2a-1faa-4925-93b4-e7149617fab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:35.245719 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-f90f9219-42af-4a12-aaf4-bc345d0e5e19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:35.245719 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:35.245719 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:35.245719 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:35.245719 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:35.245719 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:35.245719 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:35.245719 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:35.245719 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:35.245719 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:35.245719 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:35.245719 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:35.245719 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:35.245719 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:35.245719 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:35.246562 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:35.246562 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:35.246562 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:35.246562 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:35.246562 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:35.246562 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:35.246562 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:35.246562 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:35.246562 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:35.246562 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:35.246562 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:35.246562 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:35.246562 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:35.246562 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:35.246562 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:35.246562 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:35.246562 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:35.247415 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:35.247415 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:35.247415 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:35.247415 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:35.247640 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396be2a-1faa-4925-93b4-e7149617fab6 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:35.247640 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396be2a-1faa-4925-93b4-e7149617fab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:35.247640 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396be2a-1faa-4925-93b4-e7149617fab6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:35.247814 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 387/805] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:35.248354 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f9219-42af-4a12-aaf4-bc345d0e5e19 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:35.263564 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1396be2a-1faa-4925-93b4-e7149617fab6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:56:35.268894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-1396be2a-1faa-4925-93b4-e7149617fab6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:56:35.268894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:35.268894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:35.268894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:35.268894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:35.268894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:35.268894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:35.268894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:35.268894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:35.268894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:35.268894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:35.268894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:35.268894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:56:35.268894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:56:35.268894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:56:35.268894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:56:35.269825 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:56:35.269825 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:56:35.269825 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:56:35.269825 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:56:35.269825 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:35.269825 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:35.269825 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:35.269825 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:35.269825 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:35.269825 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:35.269825 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:35.269825 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:35.269825 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:35.269825 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:35.269825 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:35.269825 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:35.270633 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:35.270633 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:35.270633 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:35.270633 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:35.270633 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:56:35.270633 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:56:35.270633 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:56:35.270633 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:56:35.270633 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:56:35.270633 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:56:35.270633 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:56:35.270633 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:35.270633 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 419/806] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 13:56:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:35.271308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1396be2a-1faa-4925-93b4-e7149617fab6 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:35.346625 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:35.387439 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f9219-42af-4a12-aaf4-bc345d0e5e19 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:35.416422 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1396be2a-1faa-4925-93b4-e7149617fab6 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:35.425813 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9fcc8a-6d9b-4086-96d3-35ab0691b335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:35.426775 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9fcc8a-6d9b-4086-96d3-35ab0691b335 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:35.427255 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9fcc8a-6d9b-4086-96d3-35ab0691b335 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:35.427721 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9fcc8a-6d9b-4086-96d3-35ab0691b335 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:35.429414 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9fcc8a-6d9b-4086-96d3-35ab0691b335 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:35.430981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9fcc8a-6d9b-4086-96d3-35ab0691b335 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:35.457809 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 388/807] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 13:56:35 2022] POST /identity/v3/endpoints => generated 346 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:56:35.458996 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9fcc8a-6d9b-4086-96d3-35ab0691b335 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:35.488185 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade12976-790b-4e77-8942-c59e5c030360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:35.489324 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade12976-790b-4e77-8942-c59e5c030360 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:35.491339 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade12976-790b-4e77-8942-c59e5c030360 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:35.491908 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade12976-790b-4e77-8942-c59e5c030360 None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:35.492905 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade12976-790b-4e77-8942-c59e5c030360 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:35.493466 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade12976-790b-4e77-8942-c59e5c030360 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:35.495479 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade12976-790b-4e77-8942-c59e5c030360 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:35.511782 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 420/808] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:35.513185 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a53aebc-8f84-402c-a85d-61971c418f4e None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:35.513484 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a53aebc-8f84-402c-a85d-61971c418f4e None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:35.513764 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a53aebc-8f84-402c-a85d-61971c418f4e None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:35.519306 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 421/809] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:35.520194 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a53aebc-8f84-402c-a85d-61971c418f4e None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:35.533370 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:35.622874 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9fcc8a-6d9b-4086-96d3-35ab0691b335 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:35.650197 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a53aebc-8f84-402c-a85d-61971c418f4e None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:35.653161 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:35.667849 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f9e72-ce8c-4724-ad3b-3cfabc6eaa76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:35.668418 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f9e72-ce8c-4724-ad3b-3cfabc6eaa76 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:35.668733 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f9e72-ce8c-4724-ad3b-3cfabc6eaa76 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:35.669011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f9e72-ce8c-4724-ad3b-3cfabc6eaa76 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:35.669745 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f9e72-ce8c-4724-ad3b-3cfabc6eaa76 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:35.670055 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f9e72-ce8c-4724-ad3b-3cfabc6eaa76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:35.671501 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f9e72-ce8c-4724-ad3b-3cfabc6eaa76 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:35.688427 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 389/810] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:56:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:35.688427 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade08c36-2499-466e-9ed4-4d6007474da8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:35.688427 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade08c36-2499-466e-9ed4-4d6007474da8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:35.688923 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade08c36-2499-466e-9ed4-4d6007474da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:35.696834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f886d54-36c8-46cb-8fa0-56b2dc8289a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:35.697488 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f886d54-36c8-46cb-8fa0-56b2dc8289a2 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:35.697773 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f886d54-36c8-46cb-8fa0-56b2dc8289a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:35.698053 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f886d54-36c8-46cb-8fa0-56b2dc8289a2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:35.707518 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f886d54-36c8-46cb-8fa0-56b2dc8289a2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:35.709535 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f886d54-36c8-46cb-8fa0-56b2dc8289a2 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:35.709750 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ade08c36-2499-466e-9ed4-4d6007474da8 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:35.710952 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ade08c36-2499-466e-9ed4-4d6007474da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:35.723505 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-9f886d54-36c8-46cb-8fa0-56b2dc8289a2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 13:56:35.723505 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:35.723505 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:35.723505 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:35.723505 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:35.723505 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:35.723505 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:35.723505 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:35.723505 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:35.723505 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:35.723505 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:35.723505 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:35.723505 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:56:35.723505 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:56:35.723505 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:56:35.723505 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:56:35.724472 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:35.724472 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:35.724472 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:56:35.724472 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:35.724472 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:56:35.724472 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:35.724472 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:35.724472 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:35.724472 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:35.724472 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:35.724472 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:35.724472 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:35.724472 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:35.724472 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:35.724472 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:35.724472 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:35.724472 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:35.724472 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:35.726451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:35.726451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:35.726451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:56:35.726451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:56:35.726451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:56:35.726451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:56:35.726451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:56:35.726451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:56:35.726451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 13:56:35.726451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:35.726451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 422/811] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 1 13:56:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:35.726451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f886d54-36c8-46cb-8fa0-56b2dc8289a2 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:35.804909 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 390/812] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:56:35.805784 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade08c36-2499-466e-9ed4-4d6007474da8 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:35.850724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:35.852997 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f886d54-36c8-46cb-8fa0-56b2dc8289a2 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:35.891613 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ffbd70-f3b5-4cc0-a3b9-895779e771a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:35.892264 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffbd70-f3b5-4cc0-a3b9-895779e771a7 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:35.892576 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffbd70-f3b5-4cc0-a3b9-895779e771a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:35.892848 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffbd70-f3b5-4cc0-a3b9-895779e771a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:35.905299 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ffbd70-f3b5-4cc0-a3b9-895779e771a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:35.906748 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ffbd70-f3b5-4cc0-a3b9-895779e771a7 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:35.923138 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-f9ffbd70-f3b5-4cc0-a3b9-895779e771a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:35.923138 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:35.923138 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:35.923138 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:35.923138 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:35.923138 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:35.923138 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:35.923138 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:35.923138 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:35.923138 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:35.923138 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:35.923138 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:35.923138 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:35.923138 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:35.923138 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:35.924195 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:35.924195 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:35.924195 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:35.924195 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:35.924195 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:35.924195 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:35.924195 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:35.924195 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:35.924195 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:35.924195 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:35.924195 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:35.924195 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:35.924195 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:35.924195 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:35.924195 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:35.924195 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:35.924195 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:35.925161 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:35.925161 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:35.925161 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:35.925161 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:35.925161 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 423/813] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:35.925436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ffbd70-f3b5-4cc0-a3b9-895779e771a7 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:35.946003 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:35.949317 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade08c36-2499-466e-9ed4-4d6007474da8 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:35.990610 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42c254-a0f9-4e89-bd55-eb954890f81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:35.991238 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42c254-a0f9-4e89-bd55-eb954890f81f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:35.991523 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42c254-a0f9-4e89-bd55-eb954890f81f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:35.991804 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42c254-a0f9-4e89-bd55-eb954890f81f None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:35.992649 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e42c254-a0f9-4e89-bd55-eb954890f81f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:35.993016 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e42c254-a0f9-4e89-bd55-eb954890f81f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:35.994499 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e42c254-a0f9-4e89-bd55-eb954890f81f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:36.013385 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 391/814] 10.222.0.48 () {56 vars in 1149 bytes} [Mon Aug 1 13:56:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:36.014223 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42c254-a0f9-4e89-bd55-eb954890f81f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:36.056624 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ffbd70-f3b5-4cc0-a3b9-895779e771a7 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:36.106243 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:36.111834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2b2e7-ac46-4f7d-bff8-a9a6e9d92c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:36.112506 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2b2e7-ac46-4f7d-bff8-a9a6e9d92c4a None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:36.112799 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2b2e7-ac46-4f7d-bff8-a9a6e9d92c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:36.113071 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2b2e7-ac46-4f7d-bff8-a9a6e9d92c4a None admin] PATH_INFO: `/v3/services/network` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:36.113711 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf2b2e7-ac46-4f7d-bff8-a9a6e9d92c4a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:36.115082 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf2b2e7-ac46-4f7d-bff8-a9a6e9d92c4a None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:36.127504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-4bf2b2e7-ac46-4f7d-bff8-a9a6e9d92c4a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 13:56:36.127504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:36.127504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:56:36.127504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:56:36.127504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:56:36.127504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:56:36.127504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:56:36.127504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:36.127504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 13:56:36.127504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:36.127504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:56:36.127504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:36.127504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:36.127504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:36.127504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:36.127504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:36.127504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:36.128523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:36.128523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:36.128523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:36.128523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:36.128523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:36.128523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:36.128523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:56:36.128523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:56:36.128523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:56:36.128523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:56:36.128523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:36.128523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:36.128523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:36.128523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:36.128523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:36.128523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:36.129360 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:36.129360 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:36.129360 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:36.129360 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:36.129360 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:36.129360 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:36.129360 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:36.129360 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:36.129360 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:36.129360 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:36.129360 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:56:36.129360 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:36.129360 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 13:56:36.129360 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:36.131626 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 424/815] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:56:35 2022] GET /identity/v3/services/network => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:36.132444 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2b2e7-ac46-4f7d-bff8-a9a6e9d92c4a None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:36.137241 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42c254-a0f9-4e89-bd55-eb954890f81f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:36.185511 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a0aab1-577f-4c6d-9baf-e619dcff5d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:36.186475 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0aab1-577f-4c6d-9baf-e619dcff5d31 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:36.186977 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0aab1-577f-4c6d-9baf-e619dcff5d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:36.187465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0aab1-577f-4c6d-9baf-e619dcff5d31 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:36.188379 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a0aab1-577f-4c6d-9baf-e619dcff5d31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:36.188894 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a0aab1-577f-4c6d-9baf-e619dcff5d31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:36.190521 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a0aab1-577f-4c6d-9baf-e619dcff5d31 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:36.206361 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 392/816] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:36.208211 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a0aab1-577f-4c6d-9baf-e619dcff5d31 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:36.229099 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:36.258025 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2b2e7-ac46-4f7d-bff8-a9a6e9d92c4a None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:36.303889 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45fa2f2-1002-4733-9d55-1677e46e9d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:36.304700 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45fa2f2-1002-4733-9d55-1677e46e9d1b None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:36.305059 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45fa2f2-1002-4733-9d55-1677e46e9d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:36.305408 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45fa2f2-1002-4733-9d55-1677e46e9d1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:36.321290 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45fa2f2-1002-4733-9d55-1677e46e9d1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:36.323557 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45fa2f2-1002-4733-9d55-1677e46e9d1b None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:36.331227 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a0aab1-577f-4c6d-9baf-e619dcff5d31 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:36.344662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-a45fa2f2-1002-4733-9d55-1677e46e9d1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:36.344662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:36.344662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:36.344662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:36.344662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:36.344662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:36.344662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:36.344662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:36.344662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:36.344662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:36.344662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:36.344662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:36.344662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:36.344662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:36.344662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:36.345780 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:36.345780 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:36.345780 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:36.345780 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:36.345780 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:36.345780 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:36.345780 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:36.345780 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:36.345780 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:36.345780 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:36.345780 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:36.345780 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:36.345780 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:36.345780 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:36.345780 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:36.345780 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:36.345780 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:36.346875 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:36.346875 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:36.346875 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:36.346875 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:36.347632 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 425/817] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:36.348513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45fa2f2-1002-4733-9d55-1677e46e9d1b None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:36.374484 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517177c-3c4a-4640-8c9b-f9eedb51453f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:36.375452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517177c-3c4a-4640-8c9b-f9eedb51453f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:36.375920 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517177c-3c4a-4640-8c9b-f9eedb51453f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:36.376374 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517177c-3c4a-4640-8c9b-f9eedb51453f None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:36.377689 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517177c-3c4a-4640-8c9b-f9eedb51453f None admin] RBAC: Adding query filter params (name=network) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:36.378185 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517177c-3c4a-4640-8c9b-f9eedb51453f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:36.380138 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517177c-3c4a-4640-8c9b-f9eedb51453f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:36.392184 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 393/818] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 13:56:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:36.392184 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517177c-3c4a-4640-8c9b-f9eedb51453f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:36.412852 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:36.488885 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45fa2f2-1002-4733-9d55-1677e46e9d1b None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:36.529172 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:36.530325 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517177c-3c4a-4640-8c9b-f9eedb51453f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:36.536810 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69423eb-d939-48ab-ba4f-9260c04e5cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:36.537687 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69423eb-d939-48ab-ba4f-9260c04e5cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:36.538103 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69423eb-d939-48ab-ba4f-9260c04e5cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:36.538486 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69423eb-d939-48ab-ba4f-9260c04e5cb9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:36.550394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69423eb-d939-48ab-ba4f-9260c04e5cb9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:36.552687 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69423eb-d939-48ab-ba4f-9260c04e5cb9 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:36.570127 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-c69423eb-d939-48ab-ba4f-9260c04e5cb9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 13:56:36.570127 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:36.570127 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:36.570127 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:36.570127 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:36.570127 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:36.570127 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:36.570127 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:36.570127 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:36.570127 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:36.570127 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:36.570127 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:36.570127 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:56:36.570127 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:56:36.570127 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:56:36.570127 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:56:36.571298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:36.571298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:36.571298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:56:36.571298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:36.571298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:56:36.571298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:36.571298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:36.571298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:36.571298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:36.571298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:36.571298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:36.571298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:36.571298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:36.571298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:36.571298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:36.571298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:36.571298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:36.571298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:36.572418 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:36.572418 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:36.572418 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:56:36.572418 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:56:36.572418 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:56:36.572418 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:56:36.572418 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:56:36.572418 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:56:36.572418 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 13:56:36.572418 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:36.574402 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063056aa-200e-46f3-861f-8259f31d4dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:36.574546 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 426/819] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Aug 1 13:56:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:36.575462 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69423eb-d939-48ab-ba4f-9260c04e5cb9 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:36.575568 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063056aa-200e-46f3-861f-8259f31d4dfa None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:36.575970 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063056aa-200e-46f3-861f-8259f31d4dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:36.576385 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063056aa-200e-46f3-861f-8259f31d4dfa None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:36.577278 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063056aa-200e-46f3-861f-8259f31d4dfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:36.577729 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063056aa-200e-46f3-861f-8259f31d4dfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:36.580741 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063056aa-200e-46f3-861f-8259f31d4dfa None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:36.599182 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 394/820] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:36.601054 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063056aa-200e-46f3-861f-8259f31d4dfa None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:36.697577 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:36.708633 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69423eb-d939-48ab-ba4f-9260c04e5cb9 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:36.742401 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063056aa-200e-46f3-861f-8259f31d4dfa None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:36.758626 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65538032-6932-437f-bb39-1f25e8bae061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:36.759256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65538032-6932-437f-bb39-1f25e8bae061 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:36.759542 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65538032-6932-437f-bb39-1f25e8bae061 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:36.759822 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65538032-6932-437f-bb39-1f25e8bae061 None admin] PATH_INFO: `/v3/services` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:36.760532 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65538032-6932-437f-bb39-1f25e8bae061 None admin] RBAC: Adding query filter params (type=network) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:36.760847 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65538032-6932-437f-bb39-1f25e8bae061 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:36.762214 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65538032-6932-437f-bb39-1f25e8bae061 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:36.772066 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 427/821] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 13:56:36 2022] GET /identity/v3/services?type=network => generated 350 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:36.773034 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65538032-6932-437f-bb39-1f25e8bae061 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:36.786403 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a5aae-46e0-4bc5-9515-960f394637eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:36.788044 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a5aae-46e0-4bc5-9515-960f394637eb None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:36.788715 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a5aae-46e0-4bc5-9515-960f394637eb None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:36.789240 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a5aae-46e0-4bc5-9515-960f394637eb None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:36.790315 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2a5aae-46e0-4bc5-9515-960f394637eb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:36.790833 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2a5aae-46e0-4bc5-9515-960f394637eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:36.792719 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2a5aae-46e0-4bc5-9515-960f394637eb None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:36.829591 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 395/822] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Aug 1 13:56:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:36.831321 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a5aae-46e0-4bc5-9515-960f394637eb None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:36.831849 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:36.884237 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65538032-6932-437f-bb39-1f25e8bae061 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:36.919893 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06ee73-7ca1-4b7f-9f23-e3af119eb39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:36.920573 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06ee73-7ca1-4b7f-9f23-e3af119eb39f None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:36.920864 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06ee73-7ca1-4b7f-9f23-e3af119eb39f None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:36.921143 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06ee73-7ca1-4b7f-9f23-e3af119eb39f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:36.928677 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a06ee73-7ca1-4b7f-9f23-e3af119eb39f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:36.930125 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a06ee73-7ca1-4b7f-9f23-e3af119eb39f None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:36.942443 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-7a06ee73-7ca1-4b7f-9f23-e3af119eb39f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:36.942443 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:36.942443 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:36.942443 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:36.942443 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:36.942443 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:36.942443 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:36.942443 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:36.942443 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:36.942443 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:36.942443 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:36.942443 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:36.942443 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:56:36.942443 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:56:36.942443 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:56:36.943452 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:56:36.943452 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:36.943452 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:36.943452 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:36.943452 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:36.943452 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:36.943452 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:36.943452 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:36.943452 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:36.943452 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:36.943452 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:36.943452 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:36.943452 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:36.943452 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:36.943452 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:36.943452 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:36.943452 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:36.944641 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:56:36.944641 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:56:36.944641 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:56:36.944641 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:56:36.944641 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:56:36.944641 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:56:36.944641 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:36.944641 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:36.944641 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 428/823] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:56:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:36.945104 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06ee73-7ca1-4b7f-9f23-e3af119eb39f None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:36.965258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a5aae-46e0-4bc5-9515-960f394637eb None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:37.001241 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:37.011744 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3451bb9-f41d-4da8-913b-e0ea27f5c873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:37.012540 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3451bb9-f41d-4da8-913b-e0ea27f5c873 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:37.013207 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3451bb9-f41d-4da8-913b-e0ea27f5c873 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:37.013589 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3451bb9-f41d-4da8-913b-e0ea27f5c873 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:37.014395 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3451bb9-f41d-4da8-913b-e0ea27f5c873 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:37.016520 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3451bb9-f41d-4da8-913b-e0ea27f5c873 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:37.046496 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 396/824] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 13:56:36 2022] POST /identity/v3/endpoints => generated 327 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:56:37.047616 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3451bb9-f41d-4da8-913b-e0ea27f5c873 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:37.068723 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06ee73-7ca1-4b7f-9f23-e3af119eb39f None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:37.090922 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:37.124078 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246807e7-5f8e-488c-8298-e1b44bd0b518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:37.125057 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246807e7-5f8e-488c-8298-e1b44bd0b518 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:37.125492 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246807e7-5f8e-488c-8298-e1b44bd0b518 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:37.126241 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246807e7-5f8e-488c-8298-e1b44bd0b518 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:37.196519 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3451bb9-f41d-4da8-913b-e0ea27f5c873 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:37.217221 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246807e7-5f8e-488c-8298-e1b44bd0b518 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:37.219341 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246807e7-5f8e-488c-8298-e1b44bd0b518 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:37.251213 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c4a2f-7950-4948-85d7-d1c77a8fb2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:37.252174 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c4a2f-7950-4948-85d7-d1c77a8fb2cd None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:37.252407 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c4a2f-7950-4948-85d7-d1c77a8fb2cd None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:37.252711 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c4a2f-7950-4948-85d7-d1c77a8fb2cd None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:37.253482 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c4a2f-7950-4948-85d7-d1c77a8fb2cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:37.253796 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c4a2f-7950-4948-85d7-d1c77a8fb2cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:37.256393 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c4a2f-7950-4948-85d7-d1c77a8fb2cd None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:37.278559 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 397/825] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 13:56:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:37.281722 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c4a2f-7950-4948-85d7-d1c77a8fb2cd None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:37.360262 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:37.376786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 429/826] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:56:36 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 13:56:37.377889 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f1b27-2f0a-4a72-ba12-f72d2a8750cb None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:37.378181 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f1b27-2f0a-4a72-ba12-f72d2a8750cb None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:37.378459 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f1b27-2f0a-4a72-ba12-f72d2a8750cb None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:37.379688 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 430/827] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:37.385877 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f1b27-2f0a-4a72-ba12-f72d2a8750cb None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:37.402708 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:37.436996 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c4a2f-7950-4948-85d7-d1c77a8fb2cd None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:37.479847 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8195fd0-4f2a-413b-b215-ad86fd196e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:37.480569 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8195fd0-4f2a-413b-b215-ad86fd196e3f None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:37.480837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8195fd0-4f2a-413b-b215-ad86fd196e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:37.481117 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8195fd0-4f2a-413b-b215-ad86fd196e3f None admin] PATH_INFO: `/v3/projects/72ecec9fad964b1fbeaefd0519597229/users/5007a2f8e6244b0788f0a554cc572fe5/roles/d3d8169e71454005a8cf0d7544ad05dc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:37.496284 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8195fd0-4f2a-413b-b215-ad86fd196e3f None admin] RBAC: Authorizing `identity:create_grant(project_id=72ecec9fad964b1fbeaefd0519597229, user_id=5007a2f8e6244b0788f0a554cc572fe5, role_id=d3d8169e71454005a8cf0d7544ad05dc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:37.498271 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8195fd0-4f2a-413b-b215-ad86fd196e3f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:37.511592 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f1b27-2f0a-4a72-ba12-f72d2a8750cb None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:37.512258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 398/828] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 13:56:37 2022] PUT /identity/v3/projects/72ecec9fad964b1fbeaefd0519597229/users/5007a2f8e6244b0788f0a554cc572fe5/roles/d3d8169e71454005a8cf0d7544ad05dc => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 13:56:37.513306 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d9113-1766-4539-b0da-17184d80181b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:37.513594 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d9113-1766-4539-b0da-17184d80181b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:37.513870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d9113-1766-4539-b0da-17184d80181b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:37.533554 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8a9d9113-1766-4539-b0da-17184d80181b None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:37.534773 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8a9d9113-1766-4539-b0da-17184d80181b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:37.579387 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0ba91-b94e-4768-a7b4-4b2ee3706cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:37.580424 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0ba91-b94e-4768-a7b4-4b2ee3706cde None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:37.580941 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0ba91-b94e-4768-a7b4-4b2ee3706cde None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:37.581417 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0ba91-b94e-4768-a7b4-4b2ee3706cde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:37.598748 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f0ba91-b94e-4768-a7b4-4b2ee3706cde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:37.601368 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f0ba91-b94e-4768-a7b4-4b2ee3706cde None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:37.618543 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 399/829] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:37 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:56:37.622477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-55f0ba91-b94e-4768-a7b4-4b2ee3706cde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:37.622477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:37.622477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:37.622477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:37.622477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:37.622477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:37.622477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:37.622477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:37.622477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:37.622477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:37.622477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:37.622477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:37.622477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:37.622477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:37.622477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:37.623338 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:37.623338 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:37.623338 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:37.623338 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:37.623338 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:37.623338 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:37.623338 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:37.623338 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:37.623338 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:37.623338 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:37.623338 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:37.623338 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:37.623338 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:37.623338 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:37.623338 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:37.623338 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:37.623338 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:37.624201 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:37.624201 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:37.624201 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:37.624201 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:37.631591 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 431/830] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:37.632747 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d9113-1766-4539-b0da-17184d80181b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:37.643591 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0ba91-b94e-4768-a7b4-4b2ee3706cde None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:37.668286 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:37.723599 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:37.757863 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d9113-1766-4539-b0da-17184d80181b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:37.769655 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0ba91-b94e-4768-a7b4-4b2ee3706cde None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:37.800836 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3679a2-ed85-4321-95f3-de5db2f20800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:37.801479 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3679a2-ed85-4321-95f3-de5db2f20800 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:37.801763 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3679a2-ed85-4321-95f3-de5db2f20800 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:37.802046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3679a2-ed85-4321-95f3-de5db2f20800 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:37.803276 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3679a2-ed85-4321-95f3-de5db2f20800 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:37.803609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3679a2-ed85-4321-95f3-de5db2f20800 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:37.804830 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f632df42-017c-4860-93ad-61aacf3d07fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:37.805039 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3679a2-ed85-4321-95f3-de5db2f20800 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:37.805479 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f632df42-017c-4860-93ad-61aacf3d07fa None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:37.805763 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f632df42-017c-4860-93ad-61aacf3d07fa None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:37.806042 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f632df42-017c-4860-93ad-61aacf3d07fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:37.819555 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f632df42-017c-4860-93ad-61aacf3d07fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:56:37.821213 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 400/831] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:56:37.824817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-f632df42-017c-4860-93ad-61aacf3d07fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:56:37.824817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:37.824817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:37.824817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:37.824817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:37.824817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:37.824817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:37.824817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:37.824817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:37.824817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:37.824817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:37.824817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:37.824817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:56:37.824817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:56:37.824817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:56:37.824817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:56:37.825711 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:56:37.825711 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:56:37.825711 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:56:37.825711 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:56:37.825711 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:37.825711 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:37.825711 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:37.825711 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:37.825711 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:37.825711 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:37.825711 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:37.825711 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:37.825711 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:37.825711 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:37.825711 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:37.825711 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:37.826529 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:37.826529 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:37.826529 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:37.826529 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:37.826529 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:56:37.826529 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:56:37.826529 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:56:37.826529 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:56:37.826529 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:56:37.826529 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:56:37.826529 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:56:37.826529 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:37.826529 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 432/832] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 13:56:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:56:37.827194 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b6d60-b9b0-4824-a6b2-d4ef6355a686 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:37.827263 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f632df42-017c-4860-93ad-61aacf3d07fa None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:37.827330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b6d60-b9b0-4824-a6b2-d4ef6355a686 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:37.827949 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b6d60-b9b0-4824-a6b2-d4ef6355a686 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:37.836677 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 401/833] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:37.836677 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b6d60-b9b0-4824-a6b2-d4ef6355a686 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:37.923850 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:37.943110 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f632df42-017c-4860-93ad-61aacf3d07fa None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:37.944580 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b6d60-b9b0-4824-a6b2-d4ef6355a686 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:37.968914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:37.980768 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fefbfa-7067-49b3-a93b-8af5ec5c9183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:37.981404 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fefbfa-7067-49b3-a93b-8af5ec5c9183 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:37.981695 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fefbfa-7067-49b3-a93b-8af5ec5c9183 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:37.981982 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fefbfa-7067-49b3-a93b-8af5ec5c9183 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:37.982707 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fefbfa-7067-49b3-a93b-8af5ec5c9183 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:37.983022 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fefbfa-7067-49b3-a93b-8af5ec5c9183 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:37.983285 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea9f39-8799-47a2-b0ce-3f00f8dcf19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:37.984161 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea9f39-8799-47a2-b0ce-3f00f8dcf19b None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:37.984391 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fefbfa-7067-49b3-a93b-8af5ec5c9183 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:37.984884 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea9f39-8799-47a2-b0ce-3f00f8dcf19b None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:37.985363 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea9f39-8799-47a2-b0ce-3f00f8dcf19b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:37.993946 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea9f39-8799-47a2-b0ce-3f00f8dcf19b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:37.994541 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 402/834] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:56:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:56:37.995873 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f09ab-f5d8-4d48-9212-9a35a5647291 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:37.996168 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f09ab-f5d8-4d48-9212-9a35a5647291 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:37.996574 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f09ab-f5d8-4d48-9212-9a35a5647291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:37.997826 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea9f39-8799-47a2-b0ce-3f00f8dcf19b None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:38.012441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-edea9f39-8799-47a2-b0ce-3f00f8dcf19b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:38.012441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:38.012441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:38.012441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:38.012441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:38.012441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:38.012441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:38.012441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:38.012441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:38.012441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:38.012441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:38.012441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:38.012441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:56:38.012441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:56:38.012441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:56:38.013378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:56:38.013378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:38.013378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:38.013378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:38.013378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:38.013378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:38.013378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:38.013378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:38.013378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:38.013378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:38.013378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:38.013378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:38.013378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:38.013378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:38.013378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:38.013378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:38.013378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:38.014236 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:56:38.014236 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:56:38.014236 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:56:38.014236 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:56:38.014236 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:56:38.014236 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:56:38.014236 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:38.014236 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:38.015874 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 433/835] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:56:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:38.015978 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f32f09ab-f5d8-4d48-9212-9a35a5647291 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:38.016908 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea9f39-8799-47a2-b0ce-3f00f8dcf19b None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:38.017251 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f32f09ab-f5d8-4d48-9212-9a35a5647291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:38.093364 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 403/836] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:37 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:56:38.094134 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f09ab-f5d8-4d48-9212-9a35a5647291 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:38.138097 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea9f39-8799-47a2-b0ce-3f00f8dcf19b None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:38.172877 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136bf66-1a4a-413b-904e-a6059b7b6eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:38.173760 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136bf66-1a4a-413b-904e-a6059b7b6eab None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:38.174186 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136bf66-1a4a-413b-904e-a6059b7b6eab None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:38.174608 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136bf66-1a4a-413b-904e-a6059b7b6eab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:38.185992 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7136bf66-1a4a-413b-904e-a6059b7b6eab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:38.186477 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:38.188378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7136bf66-1a4a-413b-904e-a6059b7b6eab None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:38.202870 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-7136bf66-1a4a-413b-904e-a6059b7b6eab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:38.202870 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:38.202870 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:38.202870 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:38.202870 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:38.202870 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:38.202870 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:38.202870 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:38.202870 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:38.202870 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:38.202870 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:38.202870 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:38.202870 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:38.202870 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:38.202870 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:38.203891 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:38.203891 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:38.203891 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:38.203891 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:38.203891 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:38.203891 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:38.203891 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:38.203891 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:38.203891 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:38.203891 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:38.203891 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:38.203891 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:38.203891 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:38.203891 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:38.203891 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:38.203891 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:38.203891 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:38.204956 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:38.204956 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:38.204956 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:38.204956 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:38.206696 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 434/837] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:38.207623 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136bf66-1a4a-413b-904e-a6059b7b6eab None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:38.216381 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f09ab-f5d8-4d48-9212-9a35a5647291 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:38.249550 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:38.253108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14421fc6-f643-4c62-be9b-b9d58230878a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:38.253738 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14421fc6-f643-4c62-be9b-b9d58230878a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:38.254020 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14421fc6-f643-4c62-be9b-b9d58230878a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:38.254298 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14421fc6-f643-4c62-be9b-b9d58230878a None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:38.255057 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14421fc6-f643-4c62-be9b-b9d58230878a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:38.255624 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14421fc6-f643-4c62-be9b-b9d58230878a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:38.257113 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14421fc6-f643-4c62-be9b-b9d58230878a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:38.275960 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 404/838] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 13:56:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:38.275960 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14421fc6-f643-4c62-be9b-b9d58230878a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:38.335662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136bf66-1a4a-413b-904e-a6059b7b6eab None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:38.372458 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9711da6-5095-4dc7-8b16-999e67c5f0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:38.373423 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9711da6-5095-4dc7-8b16-999e67c5f0a2 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:38.373916 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9711da6-5095-4dc7-8b16-999e67c5f0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:38.374394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9711da6-5095-4dc7-8b16-999e67c5f0a2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:38.375243 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9711da6-5095-4dc7-8b16-999e67c5f0a2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:38.376826 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9711da6-5095-4dc7-8b16-999e67c5f0a2 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:38.386132 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-f9711da6-5095-4dc7-8b16-999e67c5f0a2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 13:56:38.386132 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:38.386132 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:56:38.386132 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:56:38.386132 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:56:38.386132 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:56:38.386132 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:56:38.386132 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:38.386132 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 13:56:38.386132 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:38.386132 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:56:38.386132 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:38.386132 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:38.386132 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:38.386132 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:38.386132 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:38.386132 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:38.387277 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:38.387277 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:38.387277 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:38.387277 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:38.387277 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:38.387277 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:38.387277 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:56:38.387277 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:56:38.387277 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:56:38.387277 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:56:38.387277 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:38.387277 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:38.387277 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:38.387277 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:38.387277 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:38.387277 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:38.388298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:38.388298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:38.388298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:38.388298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:38.388298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:38.388298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:38.388298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:38.388298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:38.388298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:38.388298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:38.388298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:56:38.388298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:38.388298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 13:56:38.388298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:38.389235 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14421fc6-f643-4c62-be9b-b9d58230878a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:38.395421 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 435/839] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 13:56:38 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:38.396523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9711da6-5095-4dc7-8b16-999e67c5f0a2 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:38.422981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f957f71-1232-469e-aa08-95a7249f8295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:38.423599 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f957f71-1232-469e-aa08-95a7249f8295 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:38.423883 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f957f71-1232-469e-aa08-95a7249f8295 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:38.424159 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f957f71-1232-469e-aa08-95a7249f8295 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:38.424886 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f957f71-1232-469e-aa08-95a7249f8295 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:38.425178 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f957f71-1232-469e-aa08-95a7249f8295 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:38.426525 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f957f71-1232-469e-aa08-95a7249f8295 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:38.429853 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:38.440745 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 405/840] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:38.441426 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f957f71-1232-469e-aa08-95a7249f8295 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:38.505366 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:38.509379 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9711da6-5095-4dc7-8b16-999e67c5f0a2 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:38.545426 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f957f71-1232-469e-aa08-95a7249f8295 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:38.546333 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6b390-eb8f-42f6-93d8-a344e8d41d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:38.546956 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6b390-eb8f-42f6-93d8-a344e8d41d2c None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:38.547240 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6b390-eb8f-42f6-93d8-a344e8d41d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:38.547518 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6b390-eb8f-42f6-93d8-a344e8d41d2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:38.548339 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc6b390-eb8f-42f6-93d8-a344e8d41d2c None admin] RBAC: Adding query filter params (role.id=58f91e3a13664dabb18c6227a1658fa6, scope.project.id=72ecec9fad964b1fbeaefd0519597229, user.id=a246654c783b4cc68d212e91bb05d8c6) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:38.548702 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc6b390-eb8f-42f6-93d8-a344e8d41d2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:38.550099 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc6b390-eb8f-42f6-93d8-a344e8d41d2c None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:38.559527 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 436/841] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:56:38 2022] GET /identity/v3/role_assignments?user.id=a246654c783b4cc68d212e91bb05d8c6&scope.project.id=72ecec9fad964b1fbeaefd0519597229&role.id=58f91e3a13664dabb18c6227a1658fa6 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:38.559527 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6b390-eb8f-42f6-93d8-a344e8d41d2c None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:38.594989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb10c2-c13b-4d35-9df3-bdfe260fa8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:38.595903 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb10c2-c13b-4d35-9df3-bdfe260fa8f7 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:38.596186 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb10c2-c13b-4d35-9df3-bdfe260fa8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:38.596463 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb10c2-c13b-4d35-9df3-bdfe260fa8f7 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:38.597311 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efb10c2-c13b-4d35-9df3-bdfe260fa8f7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:38.597615 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efb10c2-c13b-4d35-9df3-bdfe260fa8f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:38.599014 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efb10c2-c13b-4d35-9df3-bdfe260fa8f7 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:38.605273 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 406/842] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 13:56:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:38.617408 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb10c2-c13b-4d35-9df3-bdfe260fa8f7 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:38.691753 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:38.705235 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6b390-eb8f-42f6-93d8-a344e8d41d2c None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:38.741949 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb10c2-c13b-4d35-9df3-bdfe260fa8f7 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:38.754418 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4848aeec-3abe-4f8a-a0c9-f60828c4e8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:38.754947 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848aeec-3abe-4f8a-a0c9-f60828c4e8e3 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:38.755507 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848aeec-3abe-4f8a-a0c9-f60828c4e8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:38.756404 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848aeec-3abe-4f8a-a0c9-f60828c4e8e3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:38.767198 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4848aeec-3abe-4f8a-a0c9-f60828c4e8e3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:38.767198 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4848aeec-3abe-4f8a-a0c9-f60828c4e8e3 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:38.782834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-4848aeec-3abe-4f8a-a0c9-f60828c4e8e3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 13:56:38.782834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:38.782834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:38.782834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:38.782834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:38.782834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:38.782834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:38.782834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:38.782834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:38.782834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:38.782834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:38.782834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:38.782834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:56:38.782834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:56:38.782834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:56:38.782834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:56:38.784833 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:38.784833 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:38.784833 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:56:38.784833 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:38.784833 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:56:38.784833 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:38.784833 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:38.784833 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:38.784833 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:38.784833 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:38.784833 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:38.784833 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:38.784833 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:38.784833 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:38.784833 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:38.784833 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:38.784833 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:38.784833 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:38.787191 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:38.787191 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:38.787191 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:56:38.787191 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:56:38.787191 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:56:38.787191 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:56:38.787191 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:56:38.787191 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:56:38.787191 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 13:56:38.787191 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:38.787191 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 437/843] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 13:56:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:38.790680 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4848aeec-3abe-4f8a-a0c9-f60828c4e8e3 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:38.804341 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab48414-f5db-44a0-a185-fe49a206ab5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:38.805006 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab48414-f5db-44a0-a185-fe49a206ab5d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:38.805289 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab48414-f5db-44a0-a185-fe49a206ab5d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:38.805563 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab48414-f5db-44a0-a185-fe49a206ab5d None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:38.806250 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab48414-f5db-44a0-a185-fe49a206ab5d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:38.806554 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab48414-f5db-44a0-a185-fe49a206ab5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:38.807922 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab48414-f5db-44a0-a185-fe49a206ab5d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:38.814328 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 407/844] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 13:56:38 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:38.830343 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:38.922700 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4848aeec-3abe-4f8a-a0c9-f60828c4e8e3 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:38.964668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b034b-5a8f-451d-a908-63afc86576b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:38.965664 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b034b-5a8f-451d-a908-63afc86576b3 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:38.966151 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b034b-5a8f-451d-a908-63afc86576b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:38.966625 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b034b-5a8f-451d-a908-63afc86576b3 None admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:38.967690 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b034b-5a8f-451d-a908-63afc86576b3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:38.968251 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b034b-5a8f-451d-a908-63afc86576b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:38.970202 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b034b-5a8f-451d-a908-63afc86576b3 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:38.999032 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 438/845] 10.222.0.48 () {56 vars in 1135 bytes} [Mon Aug 1 13:56:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:39.009270 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab48414-f5db-44a0-a185-fe49a206ab5d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:39.117531 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab48414-f5db-44a0-a185-fe49a206ab5d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:39.143000 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:39.155038 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671969f9-e2be-48f5-8892-416acbbaaf7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:39.155992 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671969f9-e2be-48f5-8892-416acbbaaf7d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:39.156513 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671969f9-e2be-48f5-8892-416acbbaaf7d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:39.156988 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671969f9-e2be-48f5-8892-416acbbaaf7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:39.232183 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671969f9-e2be-48f5-8892-416acbbaaf7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:39.233588 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671969f9-e2be-48f5-8892-416acbbaaf7d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:39.338343 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 408/846] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:56:39 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 13:56:39.345241 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b034b-5a8f-451d-a908-63afc86576b3 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:39.444773 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:39.466150 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b034b-5a8f-451d-a908-63afc86576b3 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:39.502076 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba44927-9927-4d29-8a28-1c141d38e2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:39.502649 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba44927-9927-4d29-8a28-1c141d38e2d2 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:39.502933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba44927-9927-4d29-8a28-1c141d38e2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:39.503214 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba44927-9927-4d29-8a28-1c141d38e2d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:39.514264 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba44927-9927-4d29-8a28-1c141d38e2d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:39.515747 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba44927-9927-4d29-8a28-1c141d38e2d2 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:39.531579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-8ba44927-9927-4d29-8a28-1c141d38e2d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:39.531579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:39.531579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:39.531579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:39.531579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:39.531579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:39.531579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:39.531579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:39.531579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:39.531579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:39.531579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:39.531579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:39.531579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:39.531579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:39.531579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:39.532618 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:39.532618 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:39.532618 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:39.532618 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:39.532618 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:39.532618 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:39.532618 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:39.532618 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:39.532618 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:39.532618 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:39.532618 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:39.532618 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:39.532618 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:39.532618 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:39.532618 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:39.532618 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:39.532618 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:39.533501 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:39.533501 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:39.533501 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:39.533501 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:39.533501 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 439/847] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:39.539296 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671969f9-e2be-48f5-8892-416acbbaaf7d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:39.561522 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6fa52b-04e8-4b51-af00-348427b58735 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:39.561870 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6fa52b-04e8-4b51-af00-348427b58735 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:39.562095 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6fa52b-04e8-4b51-af00-348427b58735 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:39.564517 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 440/848] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:56:39.568425 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:39.568734 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ea56f-b11f-43e9-a383-9ef6e5229182 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:39.569345 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ea56f-b11f-43e9-a383-9ef6e5229182 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:39.569345 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ea56f-b11f-43e9-a383-9ef6e5229182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:39.588910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-a54ea56f-b11f-43e9-a383-9ef6e5229182 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:39.590215 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-a54ea56f-b11f-43e9-a383-9ef6e5229182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:39.669980 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671969f9-e2be-48f5-8892-416acbbaaf7d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:39.676943 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 441/849] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:39 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:56:39.699898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ea56f-b11f-43e9-a383-9ef6e5229182 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:39.721361 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ebd53-9125-44ce-9dd3-30844b7aef44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:39.722644 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ebd53-9125-44ce-9dd3-30844b7aef44 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:39.723083 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ebd53-9125-44ce-9dd3-30844b7aef44 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:39.723510 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ebd53-9125-44ce-9dd3-30844b7aef44 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:39.724373 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458ebd53-9125-44ce-9dd3-30844b7aef44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:39.725387 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458ebd53-9125-44ce-9dd3-30844b7aef44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:39.727285 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458ebd53-9125-44ce-9dd3-30844b7aef44 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:39.743957 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 409/850] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:39.749912 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ebd53-9125-44ce-9dd3-30844b7aef44 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:39.782024 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:39.842966 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ea56f-b11f-43e9-a383-9ef6e5229182 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:39.855949 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:39.875604 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ebd53-9125-44ce-9dd3-30844b7aef44 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:39.878929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d5aa2f-ac52-4133-b75d-865390bb009f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:39.879897 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5aa2f-ac52-4133-b75d-865390bb009f None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:39.881206 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5aa2f-ac52-4133-b75d-865390bb009f None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:39.881683 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5aa2f-ac52-4133-b75d-865390bb009f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:39.894068 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d5aa2f-ac52-4133-b75d-865390bb009f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:56:39.899899 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-c6d5aa2f-ac52-4133-b75d-865390bb009f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:56:39.899899 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:39.899899 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:39.899899 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:39.899899 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:39.899899 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:39.899899 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:39.899899 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:39.899899 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:39.899899 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:39.899899 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:39.899899 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:39.899899 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:56:39.899899 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:56:39.899899 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:56:39.899899 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:56:39.900838 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:56:39.900838 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:56:39.900838 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:56:39.900838 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:56:39.900838 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:39.900838 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:39.900838 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:39.900838 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:39.900838 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:39.900838 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:39.900838 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:39.900838 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:39.900838 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:39.900838 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:39.900838 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:39.900838 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:39.901634 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:39.901634 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:39.901634 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:39.901634 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:39.901634 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:56:39.901634 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:56:39.901634 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:56:39.901634 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:56:39.901634 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:56:39.901634 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:56:39.901634 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:56:39.901634 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:39.904242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 442/851] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 13:56:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:56:39.911868 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d5aa2f-ac52-4133-b75d-865390bb009f None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:39.924767 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb390e-314b-454c-bc7f-6570ac4578c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:39.926680 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb390e-314b-454c-bc7f-6570ac4578c0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:39.927896 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb390e-314b-454c-bc7f-6570ac4578c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:39.928942 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb390e-314b-454c-bc7f-6570ac4578c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:39.940091 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bb390e-314b-454c-bc7f-6570ac4578c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:39.941580 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bb390e-314b-454c-bc7f-6570ac4578c0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:39.954740 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-37bb390e-314b-454c-bc7f-6570ac4578c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:39.954740 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:39.954740 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:39.954740 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:39.954740 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:39.954740 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:39.954740 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:39.954740 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:39.954740 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:39.954740 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:39.954740 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:39.954740 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:39.954740 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:56:39.954740 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:56:39.954740 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:56:39.956281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:56:39.956281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:39.956281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:39.956281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:39.956281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:39.956281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:39.956281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:39.956281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:39.956281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:39.956281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:39.956281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:39.956281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:39.956281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:39.956281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:39.956281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:39.956281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:39.956281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:39.957830 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:56:39.957830 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:56:39.957830 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:56:39.957830 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:56:39.957830 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:56:39.957830 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:56:39.957830 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:39.957830 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:39.960659 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 410/852] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:56:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:39.962513 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb390e-314b-454c-bc7f-6570ac4578c0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:40.051239 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d5aa2f-ac52-4133-b75d-865390bb009f None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:40.086255 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:40.101115 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e64460-e1a2-4dd3-b2ed-b6cb7928c413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:40.101748 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e64460-e1a2-4dd3-b2ed-b6cb7928c413 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:40.102032 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e64460-e1a2-4dd3-b2ed-b6cb7928c413 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:40.102329 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e64460-e1a2-4dd3-b2ed-b6cb7928c413 None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:40.103054 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e64460-e1a2-4dd3-b2ed-b6cb7928c413 None admin] RBAC: Adding query filter params (name=service) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:40.103361 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e64460-e1a2-4dd3-b2ed-b6cb7928c413 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:40.105233 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e64460-e1a2-4dd3-b2ed-b6cb7928c413 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:40.116534 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 443/853] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 13:56:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:56:40.125724 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e64460-e1a2-4dd3-b2ed-b6cb7928c413 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:40.144471 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb390e-314b-454c-bc7f-6570ac4578c0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:40.204302 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab2a093-704f-4559-b51d-f3be4f9af6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:40.205556 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2a093-704f-4559-b51d-f3be4f9af6dc None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:40.206131 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2a093-704f-4559-b51d-f3be4f9af6dc None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:40.206694 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2a093-704f-4559-b51d-f3be4f9af6dc None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:40.208688 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab2a093-704f-4559-b51d-f3be4f9af6dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:40.209375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab2a093-704f-4559-b51d-f3be4f9af6dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:40.212825 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab2a093-704f-4559-b51d-f3be4f9af6dc None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:40.236532 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 411/854] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 13:56:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:56:40.243446 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab2a093-704f-4559-b51d-f3be4f9af6dc None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:40.250005 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e64460-e1a2-4dd3-b2ed-b6cb7928c413 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:40.250662 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:40.290384 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74ea37-9ed0-4845-a134-3a8be6fb80f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:40.291047 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74ea37-9ed0-4845-a134-3a8be6fb80f8 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:40.291331 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74ea37-9ed0-4845-a134-3a8be6fb80f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:40.291610 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74ea37-9ed0-4845-a134-3a8be6fb80f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:40.305551 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b74ea37-9ed0-4845-a134-3a8be6fb80f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:40.307354 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b74ea37-9ed0-4845-a134-3a8be6fb80f8 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:40.325798 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-0b74ea37-9ed0-4845-a134-3a8be6fb80f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:40.325798 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:40.325798 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:40.325798 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:40.325798 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:40.325798 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:40.325798 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:40.325798 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:40.325798 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:40.325798 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:40.325798 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:40.325798 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:40.325798 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:40.325798 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:40.325798 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:40.327496 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:40.327496 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:40.327496 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:40.327496 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:40.327496 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:40.327496 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:40.327496 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:40.327496 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:40.327496 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:40.327496 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:40.327496 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:40.327496 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:40.327496 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:40.327496 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:40.327496 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:40.327496 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:40.327496 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:40.329249 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:40.329249 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:40.329249 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:40.329249 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:40.329249 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 444/855] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:40.338605 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74ea37-9ed0-4845-a134-3a8be6fb80f8 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:40.403077 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab2a093-704f-4559-b51d-f3be4f9af6dc None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:40.411137 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:40.456529 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20794396-e3c3-42e3-9c03-55f0b94902c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:40.457148 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20794396-e3c3-42e3-9c03-55f0b94902c0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:40.457438 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20794396-e3c3-42e3-9c03-55f0b94902c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:40.457703 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20794396-e3c3-42e3-9c03-55f0b94902c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:40.458815 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20794396-e3c3-42e3-9c03-55f0b94902c0 None admin] RBAC: Adding query filter params (role.id=58f91e3a13664dabb18c6227a1658fa6, scope.project.id=72ecec9fad964b1fbeaefd0519597229, user.id=2fe2a6d7bd10435ba2c8e552d808b7a0) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:40.459133 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20794396-e3c3-42e3-9c03-55f0b94902c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:40.461084 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20794396-e3c3-42e3-9c03-55f0b94902c0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:40.468680 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 412/856] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:56:40 2022] GET /identity/v3/role_assignments?user.id=2fe2a6d7bd10435ba2c8e552d808b7a0&scope.project.id=72ecec9fad964b1fbeaefd0519597229&role.id=58f91e3a13664dabb18c6227a1658fa6 => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:40.545026 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74ea37-9ed0-4845-a134-3a8be6fb80f8 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:40.610377 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b9a27c-8401-4792-aef5-f9ec4c015f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:40.611825 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b9a27c-8401-4792-aef5-f9ec4c015f4e None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:40.612686 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b9a27c-8401-4792-aef5-f9ec4c015f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:40.613425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b9a27c-8401-4792-aef5-f9ec4c015f4e None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:40.615601 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b9a27c-8401-4792-aef5-f9ec4c015f4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:40.616460 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b9a27c-8401-4792-aef5-f9ec4c015f4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:40.619327 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b9a27c-8401-4792-aef5-f9ec4c015f4e None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:40.640637 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 445/857] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:56:40.649606 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20794396-e3c3-42e3-9c03-55f0b94902c0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:40.723444 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:40.767850 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20794396-e3c3-42e3-9c03-55f0b94902c0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:40.814503 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b9e11-f4b3-4b4f-904e-af3526810012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:40.816185 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b9e11-f4b3-4b4f-904e-af3526810012 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:40.816934 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b9e11-f4b3-4b4f-904e-af3526810012 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:40.817570 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b9e11-f4b3-4b4f-904e-af3526810012 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:40.831575 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7b9e11-f4b3-4b4f-904e-af3526810012 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:40.835137 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7b9e11-f4b3-4b4f-904e-af3526810012 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:40.859375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-9d7b9e11-f4b3-4b4f-904e-af3526810012 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 13:56:40.859375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:40.859375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:40.859375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:40.859375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:40.859375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:40.859375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:40.859375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:40.859375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:40.859375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:40.859375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:40.859375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:40.859375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:56:40.859375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:56:40.859375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:56:40.859375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:56:40.861321 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:40.861321 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:40.861321 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:56:40.861321 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:40.861321 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:56:40.861321 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:40.861321 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:40.861321 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:40.861321 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:40.861321 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:40.861321 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:40.861321 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:40.861321 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:40.861321 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:40.861321 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:40.861321 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:40.861321 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:40.861321 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:40.863133 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:40.863133 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:40.863133 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:56:40.863133 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:56:40.863133 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:56:40.863133 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:56:40.863133 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:56:40.863133 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:56:40.863133 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 13:56:40.863133 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:40.863133 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 413/858] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 1 13:56:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:40.871157 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b9a27c-8401-4792-aef5-f9ec4c015f4e None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:40.930834 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9ad54-5860-4491-a526-43f8fc90447e None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:40.931236 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9ad54-5860-4491-a526-43f8fc90447e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:40.931482 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9ad54-5860-4491-a526-43f8fc90447e None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:40.934008 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 414/859] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:40.940694 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d641fc-e0eb-4f52-9004-5ca2cbab023c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:40.940990 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d641fc-e0eb-4f52-9004-5ca2cbab023c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:40.941227 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d641fc-e0eb-4f52-9004-5ca2cbab023c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:40.960921 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-33d641fc-e0eb-4f52-9004-5ca2cbab023c None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:40.962103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-33d641fc-e0eb-4f52-9004-5ca2cbab023c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:41.008221 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:41.012827 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b9a27c-8401-4792-aef5-f9ec4c015f4e None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:41.044600 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 415/860] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:56:41.064525 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d641fc-e0eb-4f52-9004-5ca2cbab023c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:41.066634 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047b6bd-3578-41bf-8ae5-79ce52752e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:41.068710 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047b6bd-3578-41bf-8ae5-79ce52752e00 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:41.069212 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047b6bd-3578-41bf-8ae5-79ce52752e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:41.069713 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047b6bd-3578-41bf-8ae5-79ce52752e00 None admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:41.070747 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047b6bd-3578-41bf-8ae5-79ce52752e00 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:41.071262 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047b6bd-3578-41bf-8ae5-79ce52752e00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:41.073498 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047b6bd-3578-41bf-8ae5-79ce52752e00 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:41.100128 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 446/861] 10.222.0.48 () {56 vars in 1149 bytes} [Mon Aug 1 13:56:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:41.110175 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047b6bd-3578-41bf-8ae5-79ce52752e00 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:41.134994 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:41.190428 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d641fc-e0eb-4f52-9004-5ca2cbab023c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:41.228063 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb97b3-4b24-45a6-ba0b-cf3dfa6fd813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:41.228719 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb97b3-4b24-45a6-ba0b-cf3dfa6fd813 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:41.229005 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb97b3-4b24-45a6-ba0b-cf3dfa6fd813 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:41.229285 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb97b3-4b24-45a6-ba0b-cf3dfa6fd813 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:41.229980 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdb97b3-4b24-45a6-ba0b-cf3dfa6fd813 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:41.231355 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdb97b3-4b24-45a6-ba0b-cf3dfa6fd813 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:41.238015 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047b6bd-3578-41bf-8ae5-79ce52752e00 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:41.253011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 416/862] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 13:56:41 2022] POST /identity/v3/services => generated 251 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:56:41.286903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aadb86a-f1db-49e8-b390-6e0d240d003f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:41.287947 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aadb86a-f1db-49e8-b390-6e0d240d003f None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:41.288587 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aadb86a-f1db-49e8-b390-6e0d240d003f None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:41.289069 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aadb86a-f1db-49e8-b390-6e0d240d003f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:41.315275 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:41.340915 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cb906-8806-4609-aadb-8a54fbefaad1 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:41.341406 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cb906-8806-4609-aadb-8a54fbefaad1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:41.341951 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cb906-8806-4609-aadb-8a54fbefaad1 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:41.349030 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 417/863] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:41 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:56:41.353640 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d76a0-18f7-4923-a424-e8db9f241e54 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:41.354134 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d76a0-18f7-4923-a424-e8db9f241e54 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:41.354616 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d76a0-18f7-4923-a424-e8db9f241e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:41.370917 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aadb86a-f1db-49e8-b390-6e0d240d003f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:41.372452 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aadb86a-f1db-49e8-b390-6e0d240d003f None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:41.387684 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8b5d76a0-18f7-4923-a424-e8db9f241e54 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:41.389439 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8b5d76a0-18f7-4923-a424-e8db9f241e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:41.498094 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 447/864] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:56:41 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 13:56:41.504321 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aadb86a-f1db-49e8-b390-6e0d240d003f None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:41.505800 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 418/865] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:41 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:56:41.553546 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5d76a0-18f7-4923-a424-e8db9f241e54 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:41.573570 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:41.607282 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:41.634620 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aadb86a-f1db-49e8-b390-6e0d240d003f None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:41.670125 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5d76a0-18f7-4923-a424-e8db9f241e54 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:41.683250 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43999e17-4319-4b49-91bd-8e85dfabe782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:41.684815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43999e17-4319-4b49-91bd-8e85dfabe782 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:41.685302 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43999e17-4319-4b49-91bd-8e85dfabe782 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:41.685803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43999e17-4319-4b49-91bd-8e85dfabe782 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:41.700885 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43999e17-4319-4b49-91bd-8e85dfabe782 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:41.703552 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43999e17-4319-4b49-91bd-8e85dfabe782 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:41.707264 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa284990-8cca-46b2-9b75-4ce6b47d63bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:41.707913 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa284990-8cca-46b2-9b75-4ce6b47d63bc None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:41.708192 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa284990-8cca-46b2-9b75-4ce6b47d63bc None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:41.708462 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa284990-8cca-46b2-9b75-4ce6b47d63bc None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:41.709642 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa284990-8cca-46b2-9b75-4ce6b47d63bc None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:41.710975 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa284990-8cca-46b2-9b75-4ce6b47d63bc None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:41.719916 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-fa284990-8cca-46b2-9b75-4ce6b47d63bc None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 13:56:41.719916 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:41.719916 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:56:41.719916 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:56:41.719916 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:56:41.719916 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:56:41.719916 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:56:41.719916 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:41.719916 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 13:56:41.719916 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:41.719916 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:56:41.719916 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:41.719916 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:41.719916 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:41.719916 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:41.719916 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:41.720892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:41.720892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:41.720892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:41.720892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:41.720892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:41.720892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:41.720892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:41.720892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:56:41.720892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:56:41.720892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:56:41.720892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:56:41.720892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:41.720892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:41.720892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:41.720892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:41.720892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:41.721716 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:41.721716 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:41.721716 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:41.721716 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:41.721716 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:41.721716 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:41.721716 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:41.721716 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:41.721716 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:41.721716 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:41.721716 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:41.721716 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:56:41.721716 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:41.721716 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 13:56:41.721716 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:41.726027 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-43999e17-4319-4b49-91bd-8e85dfabe782 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:41.726027 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:41.726027 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:41.726027 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:41.726027 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:41.726027 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:41.726027 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:41.726027 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:41.726027 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:41.726027 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:41.726027 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:41.726027 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:41.726027 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:41.726027 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:41.726027 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:41.726903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:41.726903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:41.726903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:41.726903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:41.726903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:41.726903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:41.726903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:41.726903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:41.726903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:41.726903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:41.726903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:41.726903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:41.726903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:41.726903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:41.726903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:41.726903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:41.726903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:41.727770 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:41.727770 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:41.727770 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:41.727770 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:41.728002 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 419/866] 10.222.0.48 () {56 vars in 1132 bytes} [Mon Aug 1 13:56:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:41.728002 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa284990-8cca-46b2-9b75-4ce6b47d63bc None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:41.729913 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 448/867] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:56:41.736374 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43999e17-4319-4b49-91bd-8e85dfabe782 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:41.860108 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43999e17-4319-4b49-91bd-8e85dfabe782 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:41.863588 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:41.870299 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa284990-8cca-46b2-9b75-4ce6b47d63bc None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:41.889124 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:41.907428 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d044ab0-4c56-4477-ad74-0b49f53f6702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:41.909011 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d044ab0-4c56-4477-ad74-0b49f53f6702 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:41.909762 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d044ab0-4c56-4477-ad74-0b49f53f6702 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:41.910626 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d044ab0-4c56-4477-ad74-0b49f53f6702 None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:41.911903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d044ab0-4c56-4477-ad74-0b49f53f6702 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:41.912411 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d044ab0-4c56-4477-ad74-0b49f53f6702 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:41.914213 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d044ab0-4c56-4477-ad74-0b49f53f6702 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:41.915332 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a36d13-1e06-4ef2-8c91-fb49e0f79e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:41.915960 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a36d13-1e06-4ef2-8c91-fb49e0f79e9e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:41.916256 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a36d13-1e06-4ef2-8c91-fb49e0f79e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:41.916574 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a36d13-1e06-4ef2-8c91-fb49e0f79e9e None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:41.917242 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a36d13-1e06-4ef2-8c91-fb49e0f79e9e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:41.917556 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a36d13-1e06-4ef2-8c91-fb49e0f79e9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:41.918973 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a36d13-1e06-4ef2-8c91-fb49e0f79e9e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:41.924456 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 420/868] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 1 13:56:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:41.929436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a36d13-1e06-4ef2-8c91-fb49e0f79e9e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:41.935072 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 449/869] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:41.936279 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d044ab0-4c56-4477-ad74-0b49f53f6702 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:42.058609 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d044ab0-4c56-4477-ad74-0b49f53f6702 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:42.065779 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a36d13-1e06-4ef2-8c91-fb49e0f79e9e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:42.096656 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e42e7d-5fb3-40c7-a962-0002a6911eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:42.097328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e42e7d-5fb3-40c7-a962-0002a6911eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:42.097642 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e42e7d-5fb3-40c7-a962-0002a6911eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:42.097932 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e42e7d-5fb3-40c7-a962-0002a6911eb8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:42.106382 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e42e7d-5fb3-40c7-a962-0002a6911eb8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:42.107861 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e42e7d-5fb3-40c7-a962-0002a6911eb8 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:42.111962 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93426a45-b227-42db-aae5-ee8770f23d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:42.112847 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93426a45-b227-42db-aae5-ee8770f23d52 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:42.113339 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93426a45-b227-42db-aae5-ee8770f23d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:42.114070 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93426a45-b227-42db-aae5-ee8770f23d52 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:42.114983 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93426a45-b227-42db-aae5-ee8770f23d52 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:42.115499 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93426a45-b227-42db-aae5-ee8770f23d52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:42.117081 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93426a45-b227-42db-aae5-ee8770f23d52 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:42.119901 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-c9e42e7d-5fb3-40c7-a962-0002a6911eb8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:42.119901 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:42.119901 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:42.119901 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:42.119901 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:42.119901 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:42.119901 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:42.119901 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:42.119901 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:42.119901 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:42.119901 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:42.119901 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:42.119901 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:56:42.119901 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:56:42.119901 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:56:42.120788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:56:42.120788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:42.120788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:42.120788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:42.120788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:42.120788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:42.120788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:42.120788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:42.120788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:42.120788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:42.120788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:42.120788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:42.120788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:42.120788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:42.120788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:42.120788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:42.120788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:42.121637 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:56:42.121637 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:56:42.121637 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:56:42.121637 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:56:42.121637 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:56:42.121637 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:56:42.121637 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:42.121637 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:42.124813 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 421/871] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 1 13:56:41 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:56:42.125866 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 450/871] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:56:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:56:42.126859 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93426a45-b227-42db-aae5-ee8770f23d52 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:42.175488 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:42.242216 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93426a45-b227-42db-aae5-ee8770f23d52 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:42.279789 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f21135-d321-4fa0-aba0-145662a576f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:42.280415 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f21135-d321-4fa0-aba0-145662a576f5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:42.280725 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f21135-d321-4fa0-aba0-145662a576f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:42.280999 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f21135-d321-4fa0-aba0-145662a576f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:42.281752 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f21135-d321-4fa0-aba0-145662a576f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:42.282057 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f21135-d321-4fa0-aba0-145662a576f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:42.283476 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f21135-d321-4fa0-aba0-145662a576f5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:42.297151 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 422/872] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 13:56:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:42.303776 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e42e7d-5fb3-40c7-a962-0002a6911eb8 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:42.337392 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:42.416716 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e42e7d-5fb3-40c7-a962-0002a6911eb8 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:42.453875 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb01f8-c58d-453f-ba40-dd9eae87ab24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:42.454420 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb01f8-c58d-453f-ba40-dd9eae87ab24 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:42.454703 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb01f8-c58d-453f-ba40-dd9eae87ab24 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:42.454982 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb01f8-c58d-453f-ba40-dd9eae87ab24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:42.455781 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbb01f8-c58d-453f-ba40-dd9eae87ab24 None admin] RBAC: Adding query filter params (role.id=d3d8169e71454005a8cf0d7544ad05dc, scope.project.id=72ecec9fad964b1fbeaefd0519597229, user.id=5007a2f8e6244b0788f0a554cc572fe5) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:42.456089 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbb01f8-c58d-453f-ba40-dd9eae87ab24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:42.457544 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbb01f8-c58d-453f-ba40-dd9eae87ab24 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:42.464224 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 451/873] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:56:42 2022] GET /identity/v3/role_assignments?user.id=5007a2f8e6244b0788f0a554cc572fe5&scope.project.id=72ecec9fad964b1fbeaefd0519597229&role.id=d3d8169e71454005a8cf0d7544ad05dc => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:42.904548 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893f35e-6770-47f3-9a8e-f58b1bec37fa None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:42.905200 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893f35e-6770-47f3-9a8e-f58b1bec37fa None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:42.905688 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893f35e-6770-47f3-9a8e-f58b1bec37fa None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:42.907495 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 423/874] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:42.913782 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69829306-5ef6-41d4-94eb-8573d939585b None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:42.914685 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69829306-5ef6-41d4-94eb-8573d939585b None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:42.915177 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69829306-5ef6-41d4-94eb-8573d939585b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:42.942191 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-69829306-5ef6-41d4-94eb-8573d939585b None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:42.943860 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-69829306-5ef6-41d4-94eb-8573d939585b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:43.027890 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 452/875] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:42 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:56:43.050188 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2893f35e-6770-47f3-9a8e-f58b1bec37fa None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:43.155981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:43.160742 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2893f35e-6770-47f3-9a8e-f58b1bec37fa None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:43.197137 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3dece3-9b79-4fe7-beec-5c70b5c41468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:43.197780 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3dece3-9b79-4fe7-beec-5c70b5c41468 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:43.198063 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3dece3-9b79-4fe7-beec-5c70b5c41468 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:43.198336 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3dece3-9b79-4fe7-beec-5c70b5c41468 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:43.198963 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3dece3-9b79-4fe7-beec-5c70b5c41468 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:43.200317 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3dece3-9b79-4fe7-beec-5c70b5c41468 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:43.209964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-de3dece3-9b79-4fe7-beec-5c70b5c41468 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 13:56:43.209964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:43.209964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:56:43.209964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:56:43.209964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:56:43.209964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:56:43.209964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:56:43.209964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:43.209964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 13:56:43.209964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:43.209964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:56:43.209964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:43.209964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:43.209964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:43.209964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:43.209964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:43.209964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:43.210954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:43.210954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:43.210954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:43.210954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:43.210954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:43.210954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:43.210954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:56:43.210954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:56:43.210954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:56:43.210954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:56:43.210954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:43.210954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:43.210954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:43.210954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:43.210954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:43.210954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:43.211774 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:43.211774 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:43.211774 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:43.211774 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:43.211774 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:43.211774 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:43.211774 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:43.211774 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:43.211774 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:43.211774 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:43.211774 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:56:43.211774 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:43.211774 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 13:56:43.211774 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:43.211774 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 424/876] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 1 13:56:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:43.219425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69829306-5ef6-41d4-94eb-8573d939585b None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:43.239014 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:43.331149 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69829306-5ef6-41d4-94eb-8573d939585b None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:43.368839 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677e7ee-bbdb-433f-8291-3123401b0180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:43.369457 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677e7ee-bbdb-433f-8291-3123401b0180 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:43.369738 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677e7ee-bbdb-433f-8291-3123401b0180 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:43.370013 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677e7ee-bbdb-433f-8291-3123401b0180 None admin] PATH_INFO: `/v3/services` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:43.370694 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677e7ee-bbdb-433f-8291-3123401b0180 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:43.370998 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677e7ee-bbdb-433f-8291-3123401b0180 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:43.372352 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677e7ee-bbdb-433f-8291-3123401b0180 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:43.377605 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 453/877] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 1 13:56:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:43.384137 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3dece3-9b79-4fe7-beec-5c70b5c41468 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:43.448619 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62606e56-f0e2-40af-90ba-3d7a77262164 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:43.448933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62606e56-f0e2-40af-90ba-3d7a77262164 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:43.449224 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62606e56-f0e2-40af-90ba-3d7a77262164 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:43.452395 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 454/878] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:56:43.456350 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c12c71-e065-4980-9877-7266ac96eb9c None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:43.456898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c12c71-e065-4980-9877-7266ac96eb9c None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:43.457380 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c12c71-e065-4980-9877-7266ac96eb9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:43.482778 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-d8c12c71-e065-4980-9877-7266ac96eb9c None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:43.484316 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-d8c12c71-e065-4980-9877-7266ac96eb9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:43.495774 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3dece3-9b79-4fe7-beec-5c70b5c41468 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:43.531780 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af6f158-baf0-4fa7-8a0c-52081eb1eb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:43.532398 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6f158-baf0-4fa7-8a0c-52081eb1eb31 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:43.532706 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6f158-baf0-4fa7-8a0c-52081eb1eb31 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:43.532981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6f158-baf0-4fa7-8a0c-52081eb1eb31 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:43.533653 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af6f158-baf0-4fa7-8a0c-52081eb1eb31 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:43.533953 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af6f158-baf0-4fa7-8a0c-52081eb1eb31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:43.535310 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af6f158-baf0-4fa7-8a0c-52081eb1eb31 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:43.540516 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 425/879] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 1 13:56:43 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:43.580625 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 455/880] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:43 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:56:43.596650 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af6f158-baf0-4fa7-8a0c-52081eb1eb31 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:43.629502 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:43.719661 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af6f158-baf0-4fa7-8a0c-52081eb1eb31 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:43.754820 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f97df7-057d-436d-b5fc-23822e446da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:43.756255 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f97df7-057d-436d-b5fc-23822e446da8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:43.756564 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f97df7-057d-436d-b5fc-23822e446da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:43.756842 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f97df7-057d-436d-b5fc-23822e446da8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:43.757473 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f97df7-057d-436d-b5fc-23822e446da8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:43.758839 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f97df7-057d-436d-b5fc-23822e446da8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:43.767981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-a2f97df7-057d-436d-b5fc-23822e446da8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 13:56:43.767981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:43.767981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:56:43.767981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:56:43.767981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:56:43.767981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:56:43.767981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:56:43.767981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:43.767981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 13:56:43.767981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:43.767981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:56:43.767981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:43.767981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:43.767981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:43.767981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:43.767981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:43.767981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:43.769062 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:43.769062 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:43.769062 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:43.769062 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:43.769062 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:43.769062 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:43.769062 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:56:43.769062 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:56:43.769062 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:56:43.769062 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:56:43.769062 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:43.769062 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:43.769062 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:43.769062 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:43.769062 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:43.769062 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:43.769888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:43.769888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:43.769888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:43.769888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:43.769888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:43.769888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:43.769888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:43.769888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:43.769888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:43.769888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:43.769888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:56:43.769888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:43.769888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 13:56:43.769888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:43.769888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 426/881] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 13:56:43 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:43.775723 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c12c71-e065-4980-9877-7266ac96eb9c None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:43.789818 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:43.881148 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c12c71-e065-4980-9877-7266ac96eb9c None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:43.914712 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec34059-e2ef-4b92-874a-b4ee0cde4e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:43.915324 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec34059-e2ef-4b92-874a-b4ee0cde4e20 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:43.915601 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec34059-e2ef-4b92-874a-b4ee0cde4e20 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:43.915877 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec34059-e2ef-4b92-874a-b4ee0cde4e20 None admin] PATH_INFO: `/v3/services` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:43.916584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec34059-e2ef-4b92-874a-b4ee0cde4e20 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:43.916885 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec34059-e2ef-4b92-874a-b4ee0cde4e20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:43.918239 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec34059-e2ef-4b92-874a-b4ee0cde4e20 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:43.928012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 456/882] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 13:56:43 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:43.929370 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f97df7-057d-436d-b5fc-23822e446da8 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:44.032972 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:44.044184 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f97df7-057d-436d-b5fc-23822e446da8 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:44.082830 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f8bbd-8cf1-481c-9a45-0f575773cc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:44.083872 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f8bbd-8cf1-481c-9a45-0f575773cc43 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:44.084404 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f8bbd-8cf1-481c-9a45-0f575773cc43 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:44.084996 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f8bbd-8cf1-481c-9a45-0f575773cc43 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:44.085918 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3f8bbd-8cf1-481c-9a45-0f575773cc43 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:44.086433 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3f8bbd-8cf1-481c-9a45-0f575773cc43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:44.088019 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3f8bbd-8cf1-481c-9a45-0f575773cc43 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:44.093978 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 427/883] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 13:56:43 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:44.099688 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec34059-e2ef-4b92-874a-b4ee0cde4e20 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:44.181186 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:44.205658 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec34059-e2ef-4b92-874a-b4ee0cde4e20 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:44.245891 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748738b-ffef-4f7f-905a-39e68303f386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:44.246520 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748738b-ffef-4f7f-905a-39e68303f386 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:44.246806 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748738b-ffef-4f7f-905a-39e68303f386 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:44.247083 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748738b-ffef-4f7f-905a-39e68303f386 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:44.247850 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9748738b-ffef-4f7f-905a-39e68303f386 None admin] RBAC: Adding query filter params (interface=public, service_id=e7c3d090d35b44f1b4b8e7e4c70f113b, region_id=RegionOne) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:44.248161 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9748738b-ffef-4f7f-905a-39e68303f386 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:44.249559 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9748738b-ffef-4f7f-905a-39e68303f386 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:44.255633 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 457/884] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Aug 1 13:56:44 2022] GET /identity/v3/endpoints?service_id=e7c3d090d35b44f1b4b8e7e4c70f113b&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:44.260868 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f8bbd-8cf1-481c-9a45-0f575773cc43 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:44.321467 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411cc2d-af26-4c41-97cb-48a0008e3212 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:44.322174 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411cc2d-af26-4c41-97cb-48a0008e3212 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:44.322692 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411cc2d-af26-4c41-97cb-48a0008e3212 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:44.324660 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 458/885] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:44.329709 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee825b-b9d7-4531-b9ad-b51e6c3bcd9c None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:44.329991 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee825b-b9d7-4531-b9ad-b51e6c3bcd9c None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:44.330265 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee825b-b9d7-4531-b9ad-b51e6c3bcd9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:44.349617 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-e3ee825b-b9d7-4531-b9ad-b51e6c3bcd9c None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:44.350823 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-e3ee825b-b9d7-4531-b9ad-b51e6c3bcd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:44.396528 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f8bbd-8cf1-481c-9a45-0f575773cc43 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:44.448528 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 459/886] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:44 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:56:44.448722 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff999e41-c24d-4ba8-b7c4-ada79606c717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:44.448722 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff999e41-c24d-4ba8-b7c4-ada79606c717 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:44.448722 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff999e41-c24d-4ba8-b7c4-ada79606c717 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:44.448722 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff999e41-c24d-4ba8-b7c4-ada79606c717 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:44.449026 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff999e41-c24d-4ba8-b7c4-ada79606c717 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:44.449630 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff999e41-c24d-4ba8-b7c4-ada79606c717 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:44.460644 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 428/887] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Aug 1 13:56:44 2022] GET /identity/v3/services => generated 1011 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 13:56:44.465075 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee825b-b9d7-4531-b9ad-b51e6c3bcd9c None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:44.543858 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:44.578239 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee825b-b9d7-4531-b9ad-b51e6c3bcd9c None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:44.588905 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df62272-11c9-4bbf-9226-53ba1212561d None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:44.589251 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df62272-11c9-4bbf-9226-53ba1212561d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:44.589470 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df62272-11c9-4bbf-9226-53ba1212561d None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:44.591103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 429/888] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:56:44.596507 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854bc53-f6f8-4551-a9dd-76484d45323c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:44.596805 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854bc53-f6f8-4551-a9dd-76484d45323c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:44.597092 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854bc53-f6f8-4551-a9dd-76484d45323c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:44.616662 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-2854bc53-f6f8-4551-a9dd-76484d45323c None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:44.617992 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2854bc53-f6f8-4551-a9dd-76484d45323c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:44.623327 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1c02b-146b-4d83-99fe-afa7f1f62868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:44.624299 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1c02b-146b-4d83-99fe-afa7f1f62868 None admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:44.624839 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1c02b-146b-4d83-99fe-afa7f1f62868 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:44.625328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1c02b-146b-4d83-99fe-afa7f1f62868 None admin] PATH_INFO: `/v3/services` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:44.626264 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf1c02b-146b-4d83-99fe-afa7f1f62868 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:44.627908 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf1c02b-146b-4d83-99fe-afa7f1f62868 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:44.634880 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:44.652703 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 460/889] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 13:56:44 2022] POST /identity/v3/services => generated 269 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:56:44.718410 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 430/890] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:44 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:56:44.749004 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1c02b-146b-4d83-99fe-afa7f1f62868 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:44.870555 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1c02b-146b-4d83-99fe-afa7f1f62868 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:44.903234 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:44.909971 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e0131-a52f-4c0c-a1e8-6c9bdbdc88b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:44.911151 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e0131-a52f-4c0c-a1e8-6c9bdbdc88b2 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:44.911665 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e0131-a52f-4c0c-a1e8-6c9bdbdc88b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:44.911944 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e0131-a52f-4c0c-a1e8-6c9bdbdc88b2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:44.923335 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e0131-a52f-4c0c-a1e8-6c9bdbdc88b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:56:44.929534 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-a25e0131-a52f-4c0c-a1e8-6c9bdbdc88b2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 13:56:44.929534 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:44.929534 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:44.929534 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:44.929534 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:44.929534 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:44.929534 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:44.929534 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:44.929534 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:44.929534 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:44.929534 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:44.929534 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:44.929534 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:56:44.929534 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:56:44.929534 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:56:44.929534 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:56:44.930432 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:56:44.930432 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:56:44.930432 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:56:44.930432 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:56:44.930432 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:44.930432 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:44.930432 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:44.930432 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:44.930432 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:44.930432 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:44.930432 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:44.930432 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:44.930432 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:44.930432 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:44.930432 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:44.930432 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:44.931243 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:44.931243 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:44.931243 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:44.931243 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:44.931243 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:56:44.931243 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:56:44.931243 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:56:44.931243 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:56:44.931243 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:56:44.931243 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:56:44.931243 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 13:56:44.931243 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:44.931243 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 461/891] 10.222.0.48 () {56 vars in 1120 bytes} [Mon Aug 1 13:56:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:44.936650 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854bc53-f6f8-4551-a9dd-76484d45323c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:45.037520 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854bc53-f6f8-4551-a9dd-76484d45323c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:45.077406 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b939196-9050-433d-a5dd-4e7e2f0ca67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:45.078035 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b939196-9050-433d-a5dd-4e7e2f0ca67a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:45.078319 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b939196-9050-433d-a5dd-4e7e2f0ca67a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:45.078599 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b939196-9050-433d-a5dd-4e7e2f0ca67a None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:45.079322 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b939196-9050-433d-a5dd-4e7e2f0ca67a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:45.079630 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b939196-9050-433d-a5dd-4e7e2f0ca67a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:45.081026 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b939196-9050-433d-a5dd-4e7e2f0ca67a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:45.090917 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 431/892] 10.222.0.48 () {56 vars in 1115 bytes} [Mon Aug 1 13:56:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:45.097570 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e0131-a52f-4c0c-a1e8-6c9bdbdc88b2 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:45.201893 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e0131-a52f-4c0c-a1e8-6c9bdbdc88b2 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:45.237330 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807971a4-175a-4fdc-9e52-0d6e4e73d55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:45.238221 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807971a4-175a-4fdc-9e52-0d6e4e73d55b None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:45.238841 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807971a4-175a-4fdc-9e52-0d6e4e73d55b None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:45.239284 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807971a4-175a-4fdc-9e52-0d6e4e73d55b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:45.250451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807971a4-175a-4fdc-9e52-0d6e4e73d55b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:45.252076 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807971a4-175a-4fdc-9e52-0d6e4e73d55b None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:45.267698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-807971a4-175a-4fdc-9e52-0d6e4e73d55b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:45.267698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:45.267698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:45.267698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:45.267698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:45.267698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:45.267698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:45.267698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:45.267698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:45.267698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:45.267698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:45.267698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:45.267698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:45.267698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:45.267698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:45.268852 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:45.268852 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:45.268852 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:45.268852 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:45.268852 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:45.268852 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:45.268852 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:45.268852 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:45.268852 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:45.268852 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:45.268852 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:45.268852 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:45.268852 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:45.268852 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:45.268852 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:45.268852 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:45.268852 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:45.269959 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:45.269959 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:45.269959 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:45.269959 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:45.272367 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 462/893] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:45.278206 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b939196-9050-433d-a5dd-4e7e2f0ca67a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:45.290751 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:45.387758 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b939196-9050-433d-a5dd-4e7e2f0ca67a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:45.422885 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e6080-8f7f-42e4-971f-856eba5ec0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:45.423493 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e6080-8f7f-42e4-971f-856eba5ec0cc None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:45.423727 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e6080-8f7f-42e4-971f-856eba5ec0cc None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:45.423987 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e6080-8f7f-42e4-971f-856eba5ec0cc None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:45.424718 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e6080-8f7f-42e4-971f-856eba5ec0cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:45.425031 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e6080-8f7f-42e4-971f-856eba5ec0cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:45.426393 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e6080-8f7f-42e4-971f-856eba5ec0cc None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:45.440085 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 432/894] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:45.446261 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807971a4-175a-4fdc-9e52-0d6e4e73d55b None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:45.491884 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:45.555239 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807971a4-175a-4fdc-9e52-0d6e4e73d55b None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:45.594577 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29076bdb-3633-4788-9b14-2a186c5f86b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:45.595218 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29076bdb-3633-4788-9b14-2a186c5f86b5 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:45.595499 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29076bdb-3633-4788-9b14-2a186c5f86b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:45.595774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29076bdb-3633-4788-9b14-2a186c5f86b5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:45.603305 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29076bdb-3633-4788-9b14-2a186c5f86b5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:45.604773 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29076bdb-3633-4788-9b14-2a186c5f86b5 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:45.616425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-29076bdb-3633-4788-9b14-2a186c5f86b5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 13:56:45.616425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:45.616425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:45.616425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:45.616425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:45.616425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:45.616425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:45.616425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:45.616425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:45.616425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:45.616425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:45.616425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:45.616425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:56:45.616425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:56:45.616425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:56:45.616425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:56:45.617385 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:45.617385 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:45.617385 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:56:45.617385 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:45.617385 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:56:45.617385 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:45.617385 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:45.617385 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:45.617385 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:45.617385 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:45.617385 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:45.617385 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:45.617385 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:45.617385 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:45.617385 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:45.617385 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:45.617385 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:45.617385 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:45.618968 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:45.618968 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:45.618968 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:56:45.618968 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:56:45.618968 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:56:45.618968 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:56:45.618968 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:56:45.618968 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:56:45.618968 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 13:56:45.618968 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:45.618968 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 463/895] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 1 13:56:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:45.623302 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e6080-8f7f-42e4-971f-856eba5ec0cc None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:45.696443 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:45.735967 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e6080-8f7f-42e4-971f-856eba5ec0cc None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:45.772999 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67d480c-6ccc-47a8-9b6f-46bd3607d809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:45.773858 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d480c-6ccc-47a8-9b6f-46bd3607d809 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:45.774294 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d480c-6ccc-47a8-9b6f-46bd3607d809 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:45.774737 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d480c-6ccc-47a8-9b6f-46bd3607d809 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:45.775685 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67d480c-6ccc-47a8-9b6f-46bd3607d809 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:45.776143 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67d480c-6ccc-47a8-9b6f-46bd3607d809 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:45.777911 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67d480c-6ccc-47a8-9b6f-46bd3607d809 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:45.800625 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 433/896] 10.222.0.48 () {56 vars in 1149 bytes} [Mon Aug 1 13:56:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:45.805561 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29076bdb-3633-4788-9b14-2a186c5f86b5 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:45.820156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed327e6-a109-4c97-9645-9c7eb4634200 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:45.820437 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed327e6-a109-4c97-9645-9c7eb4634200 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:45.820750 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed327e6-a109-4c97-9645-9c7eb4634200 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:45.822333 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 434/897] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:56:45.828372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed80358-39ee-4488-a463-a96a6632e6c8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:45.828691 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed80358-39ee-4488-a463-a96a6632e6c8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:45.828970 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed80358-39ee-4488-a463-a96a6632e6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:45.851144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-1ed80358-39ee-4488-a463-a96a6632e6c8 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:45.852809 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1ed80358-39ee-4488-a463-a96a6632e6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:45.929061 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29076bdb-3633-4788-9b14-2a186c5f86b5 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:45.931108 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:45.942164 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 435/898] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:45 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:56:45.964662 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed80358-39ee-4488-a463-a96a6632e6c8 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:45.975795 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782d8ff8-d861-429d-9c8b-d21478cf659f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:45.976795 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d8ff8-d861-429d-9c8b-d21478cf659f None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:45.977276 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d8ff8-d861-429d-9c8b-d21478cf659f None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:45.977749 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d8ff8-d861-429d-9c8b-d21478cf659f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:46.022022 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:46.050992 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782d8ff8-d861-429d-9c8b-d21478cf659f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:46.053385 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782d8ff8-d861-429d-9c8b-d21478cf659f None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:46.069807 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed80358-39ee-4488-a463-a96a6632e6c8 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:46.106739 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c081c16-caa9-472b-a8b5-d05dc88c54a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:46.107514 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c081c16-caa9-472b-a8b5-d05dc88c54a9 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:46.107809 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c081c16-caa9-472b-a8b5-d05dc88c54a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:46.108129 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c081c16-caa9-472b-a8b5-d05dc88c54a9 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:46.109282 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c081c16-caa9-472b-a8b5-d05dc88c54a9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:46.111005 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c081c16-caa9-472b-a8b5-d05dc88c54a9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:46.133587 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 436/899] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 13:56:45 2022] POST /identity/v3/services => generated 241 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 13:56:46.177887 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 464/900] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:56:45 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 13:56:46.184111 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c081c16-caa9-472b-a8b5-d05dc88c54a9 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:46.308498 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c081c16-caa9-472b-a8b5-d05dc88c54a9 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:46.325501 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:46.354391 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c1ed15-acb4-40cd-99d1-7a0b26dcfc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:46.355045 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c1ed15-acb4-40cd-99d1-7a0b26dcfc75 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:46.355391 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c1ed15-acb4-40cd-99d1-7a0b26dcfc75 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:46.355677 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c1ed15-acb4-40cd-99d1-7a0b26dcfc75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:46.367362 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c1ed15-acb4-40cd-99d1-7a0b26dcfc75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:46.368853 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c1ed15-acb4-40cd-99d1-7a0b26dcfc75 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:46.384936 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-54c1ed15-acb4-40cd-99d1-7a0b26dcfc75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:46.384936 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:46.384936 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:46.384936 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:46.384936 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:46.384936 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:46.384936 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:46.384936 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:46.384936 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:46.384936 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:46.384936 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:46.384936 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:46.384936 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:46.384936 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:46.384936 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:46.385841 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:46.385841 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:46.385841 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:46.385841 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:46.385841 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:46.385841 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:46.385841 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:46.385841 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:46.385841 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:46.385841 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:46.385841 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:46.385841 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:46.385841 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:46.385841 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:46.385841 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:46.385841 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:46.385841 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:46.387119 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:46.387119 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:46.387119 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:46.387119 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:46.387119 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 437/901] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:46.392197 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782d8ff8-d861-429d-9c8b-d21478cf659f None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:46.394656 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:46.502678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782d8ff8-d861-429d-9c8b-d21478cf659f None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:46.541915 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6eba8-859b-459a-a215-cb38d777c754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:46.542545 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6eba8-859b-459a-a215-cb38d777c754 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:46.542884 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6eba8-859b-459a-a215-cb38d777c754 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:46.543170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6eba8-859b-459a-a215-cb38d777c754 None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:46.543875 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e6eba8-859b-459a-a215-cb38d777c754 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:46.544177 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e6eba8-859b-459a-a215-cb38d777c754 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:46.545604 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e6eba8-859b-459a-a215-cb38d777c754 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:46.557867 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 465/902] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:46.563362 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c1ed15-acb4-40cd-99d1-7a0b26dcfc75 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:46.672570 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c315f-47f2-41c3-8009-35aae25765bb None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:46.673041 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c315f-47f2-41c3-8009-35aae25765bb None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:46.673324 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c315f-47f2-41c3-8009-35aae25765bb None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:46.674924 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 466/903] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:46.675651 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c1ed15-acb4-40cd-99d1-7a0b26dcfc75 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:46.679792 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb77ec7-3b58-430d-ad4d-506a265d6abe None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:46.680080 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb77ec7-3b58-430d-ad4d-506a265d6abe None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:46.680367 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb77ec7-3b58-430d-ad4d-506a265d6abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:46.699756 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-4bb77ec7-3b58-430d-ad4d-506a265d6abe None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:46.701010 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-4bb77ec7-3b58-430d-ad4d-506a265d6abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:46.711896 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01849b0f-0f2d-4472-8140-bff3f1b55c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:46.712579 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01849b0f-0f2d-4472-8140-bff3f1b55c66 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:46.712863 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01849b0f-0f2d-4472-8140-bff3f1b55c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:46.713139 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01849b0f-0f2d-4472-8140-bff3f1b55c66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:46.720068 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01849b0f-0f2d-4472-8140-bff3f1b55c66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:46.721585 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01849b0f-0f2d-4472-8140-bff3f1b55c66 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:46.733071 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-01849b0f-0f2d-4472-8140-bff3f1b55c66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:46.733071 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:46.733071 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:46.733071 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:46.733071 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:46.733071 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:46.733071 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:46.733071 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:46.733071 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:46.733071 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:46.733071 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:46.733071 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:46.733071 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:56:46.733071 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:56:46.733071 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:56:46.734011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:56:46.734011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:46.734011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:46.734011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:46.734011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:46.734011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:46.734011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:46.734011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:46.734011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:46.734011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:46.734011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:46.734011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:46.734011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:46.734011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:46.734011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:46.734011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:46.734011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:46.734915 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:56:46.734915 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:56:46.734915 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:56:46.734915 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:56:46.734915 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:56:46.734915 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:56:46.734915 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:46.734915 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:46.734915 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 438/904] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:56:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:46.742362 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01849b0f-0f2d-4472-8140-bff3f1b55c66 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:46.769174 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:46.772401 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:46.784133 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 467/905] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:56:46.811457 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb77ec7-3b58-430d-ad4d-506a265d6abe None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:46.861537 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01849b0f-0f2d-4472-8140-bff3f1b55c66 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:46.898664 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f24cf7-b37c-47c9-b992-da55bc334567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:46.899286 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f24cf7-b37c-47c9-b992-da55bc334567 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:46.899573 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f24cf7-b37c-47c9-b992-da55bc334567 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:46.899850 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f24cf7-b37c-47c9-b992-da55bc334567 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:46.900636 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f24cf7-b37c-47c9-b992-da55bc334567 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:46.900947 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f24cf7-b37c-47c9-b992-da55bc334567 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:46.902366 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f24cf7-b37c-47c9-b992-da55bc334567 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:46.919283 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 439/906] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 13:56:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:46.919283 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc7a37-526a-4774-b900-73a565667890 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:46.919283 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc7a37-526a-4774-b900-73a565667890 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:46.919283 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc7a37-526a-4774-b900-73a565667890 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:46.921489 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 440/907] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:46.922251 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc7a37-526a-4774-b900-73a565667890 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:46.933559 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb77ec7-3b58-430d-ad4d-506a265d6abe None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:46.969668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d85d8b-b2ae-47e5-881a-07a5c31d1de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:46.970320 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d85d8b-b2ae-47e5-881a-07a5c31d1de8 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:46.970607 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d85d8b-b2ae-47e5-881a-07a5c31d1de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:46.970880 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d85d8b-b2ae-47e5-881a-07a5c31d1de8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:46.971532 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d85d8b-b2ae-47e5-881a-07a5c31d1de8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:46.972918 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d85d8b-b2ae-47e5-881a-07a5c31d1de8 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:46.981815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-08d85d8b-b2ae-47e5-881a-07a5c31d1de8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 13:56:46.981815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:46.981815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:56:46.981815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:56:46.981815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:56:46.981815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:56:46.981815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:56:46.981815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:46.981815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 13:56:46.981815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:46.981815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:56:46.981815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:46.981815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:46.981815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:46.981815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:46.981815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:46.981815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:46.982771 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:46.982771 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:46.982771 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:46.982771 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:46.982771 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:46.982771 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:46.982771 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:56:46.982771 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:56:46.982771 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:56:46.982771 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:56:46.982771 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:46.982771 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:46.982771 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:46.982771 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:46.982771 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:46.982771 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:46.983610 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:46.983610 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:46.983610 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:46.983610 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:46.983610 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:46.983610 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:46.983610 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:46.983610 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:46.983610 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:46.983610 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:46.983610 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:56:46.983610 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:46.983610 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 13:56:46.983610 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:46.983610 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 468/908] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 13:56:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:56:46.984409 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a79cd-7bda-4f42-8762-4303c51b1f54 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:46.984409 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a79cd-7bda-4f42-8762-4303c51b1f54 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:46.985112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a79cd-7bda-4f42-8762-4303c51b1f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:47.002576 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-6c5a79cd-7bda-4f42-8762-4303c51b1f54 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:47.003764 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-6c5a79cd-7bda-4f42-8762-4303c51b1f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:47.015670 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:47.035971 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc7a37-526a-4774-b900-73a565667890 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:47.061182 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:47.076220 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ddbda-04b0-40c3-80f7-1147047e1c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:47.076800 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ddbda-04b0-40c3-80f7-1147047e1c3a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:47.077084 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ddbda-04b0-40c3-80f7-1147047e1c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:47.077363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ddbda-04b0-40c3-80f7-1147047e1c3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:47.084697 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 469/909] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:56:47.084829 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359ddbda-04b0-40c3-80f7-1147047e1c3a None admin] RBAC: Adding query filter params (role.id=05973988fe294b63969752e9dc8da875, scope.project.id=72ecec9fad964b1fbeaefd0519597229, user.id=5007a2f8e6244b0788f0a554cc572fe5) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:47.084829 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359ddbda-04b0-40c3-80f7-1147047e1c3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:47.084989 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a79cd-7bda-4f42-8762-4303c51b1f54 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:47.085065 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359ddbda-04b0-40c3-80f7-1147047e1c3a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:47.091246 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 441/910] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:56:46 2022] GET /identity/v3/role_assignments?user.id=5007a2f8e6244b0788f0a554cc572fe5&scope.project.id=72ecec9fad964b1fbeaefd0519597229&role.id=05973988fe294b63969752e9dc8da875 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:47.105773 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ddbda-04b0-40c3-80f7-1147047e1c3a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:47.200757 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a79cd-7bda-4f42-8762-4303c51b1f54 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:47.245554 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd57d87e-911c-4361-acea-768e5a22ea56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:47.246192 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57d87e-911c-4361-acea-768e5a22ea56 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:47.246483 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57d87e-911c-4361-acea-768e5a22ea56 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:47.246767 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57d87e-911c-4361-acea-768e5a22ea56 None admin] PATH_INFO: `/v3/services` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:47.247461 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd57d87e-911c-4361-acea-768e5a22ea56 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:47.247773 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd57d87e-911c-4361-acea-768e5a22ea56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:47.249186 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd57d87e-911c-4361-acea-768e5a22ea56 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:47.255333 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 470/911] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 13:56:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:47.257574 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ddbda-04b0-40c3-80f7-1147047e1c3a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:47.261388 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd57d87e-911c-4361-acea-768e5a22ea56 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:47.300171 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998df11f-85e8-4908-ad82-dd5cf7c258cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:47.300171 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998df11f-85e8-4908-ad82-dd5cf7c258cd None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:47.300171 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998df11f-85e8-4908-ad82-dd5cf7c258cd None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:47.300171 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998df11f-85e8-4908-ad82-dd5cf7c258cd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:47.300605 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998df11f-85e8-4908-ad82-dd5cf7c258cd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:47.300605 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998df11f-85e8-4908-ad82-dd5cf7c258cd None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:47.307575 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-998df11f-85e8-4908-ad82-dd5cf7c258cd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 13:56:47.307575 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:47.307575 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:56:47.307575 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:56:47.307575 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:56:47.307575 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:56:47.307575 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:56:47.307575 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:47.307575 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 13:56:47.307575 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:47.307575 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:56:47.307575 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:47.307575 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:47.307575 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:47.307575 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:47.307575 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:47.308506 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:47.308506 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:47.308506 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:47.308506 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:47.308506 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:47.308506 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:47.308506 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:47.308506 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:56:47.308506 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:56:47.308506 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:56:47.308506 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:56:47.308506 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:47.308506 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:47.308506 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:47.308506 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:47.308506 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:47.309344 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:47.309344 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:47.309344 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:47.309344 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:47.309344 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:47.309344 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:47.309344 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:47.309344 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:47.309344 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:47.309344 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:47.309344 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:47.309344 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:56:47.309344 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:47.309344 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 13:56:47.309344 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:47.309344 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 442/912] 10.222.0.48 () {56 vars in 1132 bytes} [Mon Aug 1 13:56:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:56:47.314780 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998df11f-85e8-4908-ad82-dd5cf7c258cd None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:47.319544 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:47.349547 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:47.374228 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd57d87e-911c-4361-acea-768e5a22ea56 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:47.409448 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129b17b6-630d-45db-a4b8-373b8f82af9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:47.410077 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b17b6-630d-45db-a4b8-373b8f82af9b None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:47.410358 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b17b6-630d-45db-a4b8-373b8f82af9b None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:47.410634 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b17b6-630d-45db-a4b8-373b8f82af9b None admin] PATH_INFO: `/v3/services` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:47.411886 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129b17b6-630d-45db-a4b8-373b8f82af9b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:47.412223 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129b17b6-630d-45db-a4b8-373b8f82af9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:47.413694 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129b17b6-630d-45db-a4b8-373b8f82af9b None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:47.419151 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 471/913] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 13:56:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:47.424652 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129b17b6-630d-45db-a4b8-373b8f82af9b None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:47.427602 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998df11f-85e8-4908-ad82-dd5cf7c258cd None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:47.463821 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587058b-2a1b-492c-8f12-078a8d6a30bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:47.464855 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587058b-2a1b-492c-8f12-078a8d6a30bf None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:47.465340 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587058b-2a1b-492c-8f12-078a8d6a30bf None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:47.465825 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587058b-2a1b-492c-8f12-078a8d6a30bf None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:47.466794 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2587058b-2a1b-492c-8f12-078a8d6a30bf None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:47.467386 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2587058b-2a1b-492c-8f12-078a8d6a30bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:47.468984 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2587058b-2a1b-492c-8f12-078a8d6a30bf None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:47.474952 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 443/914] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 1 13:56:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:47.481613 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587058b-2a1b-492c-8f12-078a8d6a30bf None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:47.553563 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129b17b6-630d-45db-a4b8-373b8f82af9b None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:47.564860 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:47.575869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:47.593401 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587058b-2a1b-492c-8f12-078a8d6a30bf None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:47.610253 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84438de7-2df0-4c35-b284-b37b25ad5f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:47.611408 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84438de7-2df0-4c35-b284-b37b25ad5f9d None admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:47.612273 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84438de7-2df0-4c35-b284-b37b25ad5f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:47.612781 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84438de7-2df0-4c35-b284-b37b25ad5f9d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:47.613773 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84438de7-2df0-4c35-b284-b37b25ad5f9d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:47.615383 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84438de7-2df0-4c35-b284-b37b25ad5f9d None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:47.631953 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d7c3fc-345a-4c64-95d6-6973ce248fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:47.632674 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7c3fc-345a-4c64-95d6-6973ce248fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:47.632912 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7c3fc-345a-4c64-95d6-6973ce248fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:47.633197 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7c3fc-345a-4c64-95d6-6973ce248fe0 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:47.633850 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d7c3fc-345a-4c64-95d6-6973ce248fe0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:47.634153 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d7c3fc-345a-4c64-95d6-6973ce248fe0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:47.635766 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d7c3fc-345a-4c64-95d6-6973ce248fe0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:47.646921 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 444/915] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 1 13:56:47 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:47.647965 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d7c3fc-345a-4c64-95d6-6973ce248fe0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:47.656507 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 472/916] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 13:56:47 2022] POST /identity/v3/endpoints => generated 346 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 13:56:47.766762 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d7c3fc-345a-4c64-95d6-6973ce248fe0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:47.815156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1ce9b6-65e7-4df0-ade6-6011e67ed9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:47.815156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ce9b6-65e7-4df0-ade6-6011e67ed9e3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:47.815156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ce9b6-65e7-4df0-ade6-6011e67ed9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:47.815156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ce9b6-65e7-4df0-ade6-6011e67ed9e3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:47.815563 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1ce9b6-65e7-4df0-ade6-6011e67ed9e3 None admin] RBAC: Adding query filter params (interface=public, service_id=b59dfa944e254b8db90bdbfe1c44ee1d, region_id=RegionOne) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:47.815563 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1ce9b6-65e7-4df0-ade6-6011e67ed9e3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:47.816376 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1ce9b6-65e7-4df0-ade6-6011e67ed9e3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:47.819159 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:47.823859 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 445/917] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Aug 1 13:56:47 2022] GET /identity/v3/endpoints?service_id=b59dfa944e254b8db90bdbfe1c44ee1d&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:47.829374 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84438de7-2df0-4c35-b284-b37b25ad5f9d None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:47.938405 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84438de7-2df0-4c35-b284-b37b25ad5f9d None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:47.976428 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc9752-ba04-4960-b591-4e3d6dd8e081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:47.977082 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc9752-ba04-4960-b591-4e3d6dd8e081 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:47.977367 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc9752-ba04-4960-b591-4e3d6dd8e081 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:47.977647 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc9752-ba04-4960-b591-4e3d6dd8e081 None admin] PATH_INFO: `/v3/services` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:47.978306 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc9752-ba04-4960-b591-4e3d6dd8e081 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:47.979731 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc9752-ba04-4960-b591-4e3d6dd8e081 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:47.986177 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 473/918] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Aug 1 13:56:47 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 13:56:48.273125 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964145cb-08f2-4966-a204-a6808b87af26 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:48.273390 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964145cb-08f2-4966-a204-a6808b87af26 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:48.273663 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964145cb-08f2-4966-a204-a6808b87af26 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:48.275215 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 446/919] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:48.280456 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a4ad7-d178-4208-94fd-b713476e912c None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:48.280786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a4ad7-d178-4208-94fd-b713476e912c None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:48.281071 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a4ad7-d178-4208-94fd-b713476e912c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:48.299639 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-587a4ad7-d178-4208-94fd-b713476e912c None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:48.300881 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-587a4ad7-d178-4208-94fd-b713476e912c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:48.330247 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:48.375820 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 474/920] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:56:48.398974 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964145cb-08f2-4966-a204-a6808b87af26 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:48.503767 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964145cb-08f2-4966-a204-a6808b87af26 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:48.540819 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f9b03-135a-4644-b211-672aac1c7839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:48.541807 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f9b03-135a-4644-b211-672aac1c7839 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:48.542308 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f9b03-135a-4644-b211-672aac1c7839 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:48.542790 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f9b03-135a-4644-b211-672aac1c7839 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:48.543609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28f9b03-135a-4644-b211-672aac1c7839 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:48.545210 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28f9b03-135a-4644-b211-672aac1c7839 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:48.555187 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-c28f9b03-135a-4644-b211-672aac1c7839 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 13:56:48.555187 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:48.555187 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:56:48.555187 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:56:48.555187 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:56:48.555187 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:56:48.555187 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:56:48.555187 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:48.555187 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 13:56:48.555187 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:48.555187 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:56:48.555187 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:48.555187 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:48.555187 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:48.555187 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:48.555187 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:48.555187 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:48.556246 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:48.556246 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:48.556246 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:48.556246 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:48.556246 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:48.556246 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:48.556246 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:56:48.556246 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:56:48.556246 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:56:48.556246 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:56:48.556246 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:48.556246 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:48.556246 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:48.556246 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:48.556246 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:48.556246 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:48.557161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:48.557161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:48.557161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:48.557161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:48.557161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:48.557161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:48.557161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:48.557161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:48.557161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:48.557161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:48.557161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:56:48.557161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:48.557161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 13:56:48.557161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:48.560096 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 447/921] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 1 13:56:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:48.565799 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a4ad7-d178-4208-94fd-b713476e912c None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:48.675474 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a4ad7-d178-4208-94fd-b713476e912c None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:48.711787 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bbb6e7-d462-4c81-8a40-3b36ead02380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:48.712417 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbb6e7-d462-4c81-8a40-3b36ead02380 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:48.712727 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbb6e7-d462-4c81-8a40-3b36ead02380 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:48.713006 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbb6e7-d462-4c81-8a40-3b36ead02380 None admin] PATH_INFO: `/v3/services` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:48.713698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bbb6e7-d462-4c81-8a40-3b36ead02380 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:48.714004 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bbb6e7-d462-4c81-8a40-3b36ead02380 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:48.715376 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bbb6e7-d462-4c81-8a40-3b36ead02380 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:48.720544 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 475/922] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 1 13:56:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:48.726477 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f9b03-135a-4644-b211-672aac1c7839 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:48.790552 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:48.839332 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f9b03-135a-4644-b211-672aac1c7839 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:48.873942 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140cf0c5-566d-46a9-b97e-9fb014aefd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:48.874576 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140cf0c5-566d-46a9-b97e-9fb014aefd2b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:48.874859 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140cf0c5-566d-46a9-b97e-9fb014aefd2b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:48.875136 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140cf0c5-566d-46a9-b97e-9fb014aefd2b None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:48.875822 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140cf0c5-566d-46a9-b97e-9fb014aefd2b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:48.876197 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140cf0c5-566d-46a9-b97e-9fb014aefd2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:48.877618 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140cf0c5-566d-46a9-b97e-9fb014aefd2b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:48.883245 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 448/923] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 1 13:56:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:48.889771 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bbb6e7-d462-4c81-8a40-3b36ead02380 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:48.927708 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:49.007529 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bbb6e7-d462-4c81-8a40-3b36ead02380 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:49.044405 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deee0b0-913f-4880-99e0-6077bc13350c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:49.045385 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deee0b0-913f-4880-99e0-6077bc13350c None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:49.045872 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deee0b0-913f-4880-99e0-6077bc13350c None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:49.046335 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deee0b0-913f-4880-99e0-6077bc13350c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:49.047278 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deee0b0-913f-4880-99e0-6077bc13350c None admin] RBAC: Adding query filter params (interface=public, service_id=cf76751b2bcf45b1a6206435262ceddd, region_id=RegionOne) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:49.047781 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deee0b0-913f-4880-99e0-6077bc13350c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:49.049356 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deee0b0-913f-4880-99e0-6077bc13350c None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:49.055255 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 476/924] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Aug 1 13:56:48 2022] GET /identity/v3/endpoints?service_id=cf76751b2bcf45b1a6206435262ceddd&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:49.060735 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140cf0c5-566d-46a9-b97e-9fb014aefd2b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:49.150829 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51abc8-b212-4167-8a6b-27033ccee242 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:49.151158 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51abc8-b212-4167-8a6b-27033ccee242 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:49.151403 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51abc8-b212-4167-8a6b-27033ccee242 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:49.153037 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 477/925] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:49.157902 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ce9c1-ec2c-4ec0-aa88-eb33632e23ee None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:49.158189 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ce9c1-ec2c-4ec0-aa88-eb33632e23ee None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:49.158471 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ce9c1-ec2c-4ec0-aa88-eb33632e23ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:49.174336 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140cf0c5-566d-46a9-b97e-9fb014aefd2b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:49.177635 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-bc7ce9c1-ec2c-4ec0-aa88-eb33632e23ee None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:49.178375 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-bc7ce9c1-ec2c-4ec0-aa88-eb33632e23ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:49.204864 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:49.212417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed6e4e-2ec8-4b30-b6c4-42731cf627c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:49.213378 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed6e4e-2ec8-4b30-b6c4-42731cf627c2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:49.213829 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed6e4e-2ec8-4b30-b6c4-42731cf627c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:49.214257 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed6e4e-2ec8-4b30-b6c4-42731cf627c2 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:49.215049 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ed6e4e-2ec8-4b30-b6c4-42731cf627c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:49.217737 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ed6e4e-2ec8-4b30-b6c4-42731cf627c2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:49.225476 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 449/926] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Aug 1 13:56:49 2022] GET /identity/v3/services => generated 1497 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 13:56:49.256946 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 478/927] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:56:49.279950 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed6e4e-2ec8-4b30-b6c4-42731cf627c2 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:49.394644 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed6e4e-2ec8-4b30-b6c4-42731cf627c2 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:49.434045 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53874a42-608a-4b63-8726-8ce5370fe790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:49.435028 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53874a42-608a-4b63-8726-8ce5370fe790 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:49.435523 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53874a42-608a-4b63-8726-8ce5370fe790 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:49.435995 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53874a42-608a-4b63-8726-8ce5370fe790 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:49.447734 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53874a42-608a-4b63-8726-8ce5370fe790 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:56:49.453484 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-53874a42-608a-4b63-8726-8ce5370fe790 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 13:56:49.453484 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:49.453484 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:49.453484 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:49.453484 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:49.453484 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:49.453484 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:49.453484 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:49.453484 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:49.453484 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:49.453484 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:49.453484 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:49.453484 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:56:49.453484 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:56:49.453484 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:56:49.453484 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:56:49.454435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:56:49.454435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:56:49.454435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:56:49.454435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:56:49.454435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:49.454435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:49.454435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:49.454435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:49.454435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:49.454435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:49.454435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:49.454435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:49.454435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:49.454435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:49.454435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:49.454435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:49.455272 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:49.455272 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:49.455272 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:49.455272 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:49.455272 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:56:49.455272 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:56:49.455272 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:56:49.455272 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:56:49.455272 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:56:49.455272 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:56:49.455272 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 13:56:49.455272 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:49.458133 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 450/928] 10.222.0.48 () {56 vars in 1120 bytes} [Mon Aug 1 13:56:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:49.463704 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ce9c1-ec2c-4ec0-aa88-eb33632e23ee None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:49.489664 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:49.572358 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ce9c1-ec2c-4ec0-aa88-eb33632e23ee None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:49.608800 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c0456-462f-43ae-92af-473c56b391f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:49.609431 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c0456-462f-43ae-92af-473c56b391f7 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:49.609713 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c0456-462f-43ae-92af-473c56b391f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:49.609988 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c0456-462f-43ae-92af-473c56b391f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:49.610708 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300c0456-462f-43ae-92af-473c56b391f7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:49.611017 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300c0456-462f-43ae-92af-473c56b391f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:49.612385 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300c0456-462f-43ae-92af-473c56b391f7 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:49.621773 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 479/929] 10.222.0.48 () {56 vars in 1115 bytes} [Mon Aug 1 13:56:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:49.627397 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53874a42-608a-4b63-8726-8ce5370fe790 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:49.682878 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:49.735703 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53874a42-608a-4b63-8726-8ce5370fe790 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:49.772202 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e3e2c-38aa-40f3-970b-e936d4648e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:49.772889 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e3e2c-38aa-40f3-970b-e936d4648e1b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:49.773178 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e3e2c-38aa-40f3-970b-e936d4648e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:49.773460 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e3e2c-38aa-40f3-970b-e936d4648e1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:49.784897 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00e3e2c-38aa-40f3-970b-e936d4648e1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:49.786925 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00e3e2c-38aa-40f3-970b-e936d4648e1b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:49.808505 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-c00e3e2c-38aa-40f3-970b-e936d4648e1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:49.808505 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:49.808505 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:49.808505 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:49.808505 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:49.808505 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:49.808505 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:49.808505 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:49.808505 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:49.808505 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:49.808505 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:49.808505 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:49.808505 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:49.808505 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:49.808505 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:49.809443 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:49.809443 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:49.809443 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:49.809443 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:49.809443 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:49.809443 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:49.809443 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:49.809443 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:49.809443 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:49.809443 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:49.809443 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:49.809443 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:49.809443 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:49.809443 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:49.809443 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:49.809443 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:49.809443 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:49.810366 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:49.810366 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:49.810366 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:49.810366 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:49.810366 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 451/930] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:49.810655 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c0456-462f-43ae-92af-473c56b391f7 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:49.900692 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:49.917390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c0456-462f-43ae-92af-473c56b391f7 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:49.926835 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf8031-14b9-4c95-a8b4-09f72cdae354 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:49.927127 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf8031-14b9-4c95-a8b4-09f72cdae354 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:49.927411 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf8031-14b9-4c95-a8b4-09f72cdae354 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:49.929042 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 452/931] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:49.934220 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe2774-c361-4b5e-aa67-3080f237f10e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:49.934751 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe2774-c361-4b5e-aa67-3080f237f10e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:49.935030 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe2774-c361-4b5e-aa67-3080f237f10e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:49.954994 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de1329-3a86-4b6d-b7c2-dcc4a232d366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:49.955624 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de1329-3a86-4b6d-b7c2-dcc4a232d366 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:49.955910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de1329-3a86-4b6d-b7c2-dcc4a232d366 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:49.956189 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de1329-3a86-4b6d-b7c2-dcc4a232d366 None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:49.956917 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77de1329-3a86-4b6d-b7c2-dcc4a232d366 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:49.957224 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77de1329-3a86-4b6d-b7c2-dcc4a232d366 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:49.958682 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-bbfe2774-c361-4b5e-aa67-3080f237f10e None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:49.959225 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77de1329-3a86-4b6d-b7c2-dcc4a232d366 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:49.959873 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-bbfe2774-c361-4b5e-aa67-3080f237f10e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:49.974853 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 480/932] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:49.980271 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de1329-3a86-4b6d-b7c2-dcc4a232d366 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:50.049419 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 453/933] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:56:50.057948 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89dd7bc-899e-4889-b822-aa14cf97e22c None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:50.058238 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89dd7bc-899e-4889-b822-aa14cf97e22c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:50.058516 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89dd7bc-899e-4889-b822-aa14cf97e22c None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:50.061215 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 454/934] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:50.066260 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9219e74f-b6ee-4357-b8d8-a8745d3cfb2b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:50.066494 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9219e74f-b6ee-4357-b8d8-a8745d3cfb2b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:50.066775 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9219e74f-b6ee-4357-b8d8-a8745d3cfb2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:50.086824 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-9219e74f-b6ee-4357-b8d8-a8745d3cfb2b None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:50.087996 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9219e74f-b6ee-4357-b8d8-a8745d3cfb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:50.100583 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de1329-3a86-4b6d-b7c2-dcc4a232d366 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:50.121939 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:50.143648 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574a6874-e27e-4cf4-a69f-28c930675b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:50.144293 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a6874-e27e-4cf4-a69f-28c930675b9a None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:50.144600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a6874-e27e-4cf4-a69f-28c930675b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:50.144874 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a6874-e27e-4cf4-a69f-28c930675b9a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:50.152982 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574a6874-e27e-4cf4-a69f-28c930675b9a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:50.154632 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574a6874-e27e-4cf4-a69f-28c930675b9a None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:50.167898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-574a6874-e27e-4cf4-a69f-28c930675b9a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 13:56:50.167898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:50.167898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:50.167898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:50.167898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:50.167898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:50.167898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:50.167898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:50.167898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:50.167898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:50.167898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:50.167898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:50.167898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:56:50.167898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:56:50.167898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:56:50.167898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:56:50.169428 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:50.169428 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:50.169428 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:56:50.169428 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:50.169428 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:56:50.169428 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:50.169428 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:50.169428 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:50.169428 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:50.169428 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:50.169428 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:50.169428 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:50.169428 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:50.169428 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:50.169428 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:50.169428 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:50.169428 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:50.169428 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:50.170901 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:50.170901 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:50.170901 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:56:50.170901 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:56:50.170901 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:56:50.170901 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:56:50.170901 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:56:50.170901 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:56:50.170901 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 13:56:50.170901 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:50.174055 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 481/935] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 1 13:56:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:56:50.174720 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574a6874-e27e-4cf4-a69f-28c930675b9a None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:50.178596 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 455/936] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:56:50.179344 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9219e74f-b6ee-4357-b8d8-a8745d3cfb2b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:50.185649 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:50.286423 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9219e74f-b6ee-4357-b8d8-a8745d3cfb2b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:50.300860 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574a6874-e27e-4cf4-a69f-28c930675b9a None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:50.325282 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b838252-92a8-4dea-9f05-1496034d80be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:50.326051 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b838252-92a8-4dea-9f05-1496034d80be None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:50.326342 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b838252-92a8-4dea-9f05-1496034d80be None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:50.326624 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b838252-92a8-4dea-9f05-1496034d80be None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:50.327472 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b838252-92a8-4dea-9f05-1496034d80be None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:50.327794 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b838252-92a8-4dea-9f05-1496034d80be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:50.329250 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b838252-92a8-4dea-9f05-1496034d80be None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:50.335675 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532dadb5-adc7-4d40-84f4-6b4e22029a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:50.336457 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532dadb5-adc7-4d40-84f4-6b4e22029a18 None admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:50.336929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532dadb5-adc7-4d40-84f4-6b4e22029a18 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:50.337472 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532dadb5-adc7-4d40-84f4-6b4e22029a18 None admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:50.338420 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532dadb5-adc7-4d40-84f4-6b4e22029a18 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:50.339950 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532dadb5-adc7-4d40-84f4-6b4e22029a18 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:50.342551 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: WARNING keystone.server.flask.common [None req-532dadb5-adc7-4d40-84f4-6b4e22029a18 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 01 13:56:50.346510 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 456/937] 10.222.0.48 () {56 vars in 1149 bytes} [Mon Aug 1 13:56:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:50.347313 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b838252-92a8-4dea-9f05-1496034d80be None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:50.367731 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.sql.core [None req-532dadb5-adc7-4d40-84f4-6b4e22029a18 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 13:56:50.367731 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 01 13:56:50.367731 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [parameters: {'id': '14ed41cf39c043818ed5d4eb1427decc', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 01 13:56:50.367731 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80140) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 13:56:50.375096 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:50.376507 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-532dadb5-adc7-4d40-84f4-6b4e22029a18 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 14ed41cf39c043818ed5d4eb1427decc.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 14ed41cf39c043818ed5d4eb1427decc. Aug 01 13:56:50.376507 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:50.376507 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 13:56:50.376507 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 13:56:50.376507 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 13:56:50.376507 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 13:56:50.376507 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 13:56:50.376507 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 13:56:50.376507 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 13:56:50.376507 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application conn.query(q) Aug 01 13:56:50.376507 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 13:56:50.376507 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 13:56:50.376507 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 13:56:50.377381 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application result.read() Aug 01 13:56:50.377381 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 13:56:50.377381 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 13:56:50.377381 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 13:56:50.377381 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 13:56:50.377381 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 13:56:50.377381 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 13:56:50.377381 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 13:56:50.377381 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 13:56:50.377381 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 13:56:50.377381 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:50.377381 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 13:56:50.377381 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:50.377381 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:50.377381 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 13:56:50.377381 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 13:56:50.377381 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 01 13:56:50.377381 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 01 13:56:50.378316 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 13:56:50.378316 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application next(self.gen) Aug 01 13:56:50.378316 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 13:56:50.378316 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application yield resource Aug 01 13:56:50.378316 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 13:56:50.378316 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application next(self.gen) Aug 01 13:56:50.378316 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 13:56:50.378316 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 13:56:50.378316 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 13:56:50.378316 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 13:56:50.378316 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 13:56:50.378316 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise self.value Aug 01 13:56:50.378316 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 13:56:50.378316 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 13:56:50.378316 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 13:56:50.378316 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application session.commit() Aug 01 13:56:50.378316 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 13:56:50.379196 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 13:56:50.379196 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 13:56:50.379196 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 13:56:50.379196 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 13:56:50.379196 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self.session.flush() Aug 01 13:56:50.379196 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 13:56:50.379196 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 13:56:50.379196 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 13:56:50.379196 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 13:56:50.379196 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 13:56:50.379196 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application compat.raise_( Aug 01 13:56:50.379196 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 13:56:50.379196 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception Aug 01 13:56:50.379196 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 13:56:50.379196 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 13:56:50.379196 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 13:56:50.379196 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 13:56:50.380929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 13:56:50.380929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 13:56:50.380929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 13:56:50.380929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 13:56:50.380929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 01 13:56:50.380929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 01 13:56:50.380929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 13:56:50.380929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 13:56:50.380929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 13:56:50.380929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 13:56:50.380929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 13:56:50.380929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 13:56:50.380929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 13:56:50.380929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 13:56:50.380929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 13:56:50.381679 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 13:56:50.381679 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 13:56:50.381679 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception Aug 01 13:56:50.381679 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 13:56:50.381679 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 13:56:50.381679 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 13:56:50.381679 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 13:56:50.381679 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 13:56:50.381679 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 13:56:50.381679 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 13:56:50.381679 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application conn.query(q) Aug 01 13:56:50.381679 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 13:56:50.381679 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 13:56:50.381679 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 13:56:50.381679 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application result.read() Aug 01 13:56:50.381679 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 13:56:50.381679 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 13:56:50.382518 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 13:56:50.382518 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 13:56:50.382518 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 13:56:50.382518 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 13:56:50.382518 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 13:56:50.382518 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 13:56:50.382518 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 13:56:50.382518 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 01 13:56:50.382518 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [parameters: {'id': '14ed41cf39c043818ed5d4eb1427decc', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 01 13:56:50.382518 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 13:56:50.382518 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:50.382518 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:56:50.382518 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:50.382518 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:50.382518 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 01 13:56:50.385082 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 01 13:56:50.385082 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 13:56:50.385082 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 13:56:50.385082 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 01 13:56:50.385082 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:50.385082 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:56:50.385082 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:50.385082 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:50.385082 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:50.385082 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:50.385082 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:50.385082 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:50.385082 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:50.385082 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:50.385082 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:50.385082 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:50.385082 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:50.387594 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:50.387594 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 01 13:56:50.387594 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 01 13:56:50.387594 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:50.387594 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:50.387594 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 01 13:56:50.387594 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 01 13:56:50.387594 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 14ed41cf39c043818ed5d4eb1427decc. Aug 01 13:56:50.387594 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:50.387594 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 482/938] 10.222.0.48 () {60 vars in 1140 bytes} [Mon Aug 1 13:56:50 2022] POST /identity/v3/projects => generated 262 bytes in 207 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:50.387594 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532dadb5-adc7-4d40-84f4-6b4e22029a18 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:50.449451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:50.474396 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b838252-92a8-4dea-9f05-1496034d80be None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:50.510781 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b13e9-b7cb-460a-be94-5b84ec19da47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:50.511034 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532dadb5-adc7-4d40-84f4-6b4e22029a18 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:50.511838 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b13e9-b7cb-460a-be94-5b84ec19da47 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:50.512273 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b13e9-b7cb-460a-be94-5b84ec19da47 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:50.512795 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b13e9-b7cb-460a-be94-5b84ec19da47 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:50.513743 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765b13e9-b7cb-460a-be94-5b84ec19da47 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:50.515145 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765b13e9-b7cb-460a-be94-5b84ec19da47 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:50.528358 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-765b13e9-b7cb-460a-be94-5b84ec19da47 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 13:56:50.528358 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:50.528358 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:56:50.528358 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:56:50.528358 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:56:50.528358 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:56:50.528358 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:56:50.528358 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:50.528358 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 13:56:50.528358 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:50.528358 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:56:50.528358 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:50.528358 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:50.528358 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:50.528358 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:50.528358 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:50.529725 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:50.529725 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:50.529725 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:50.529725 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:50.529725 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:50.529725 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:50.529725 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:50.529725 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:56:50.529725 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:56:50.529725 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:56:50.529725 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:56:50.529725 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:50.529725 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:50.529725 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:50.529725 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:50.529725 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:50.530537 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:50.530537 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:50.530537 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:50.530537 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:50.530537 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:50.530537 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:50.530537 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:50.530537 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:50.530537 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:50.530537 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:50.530537 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:50.530537 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:56:50.530537 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:50.530537 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 13:56:50.530537 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:50.530537 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 457/939] 10.222.0.48 () {56 vars in 1132 bytes} [Mon Aug 1 13:56:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:50.531360 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b13e9-b7cb-460a-be94-5b84ec19da47 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:50.556780 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79e4e3-8ded-4c6b-b938-94e6ed6e26f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:50.557350 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79e4e3-8ded-4c6b-b938-94e6ed6e26f3 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:50.557630 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79e4e3-8ded-4c6b-b938-94e6ed6e26f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:50.557956 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79e4e3-8ded-4c6b-b938-94e6ed6e26f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:50.569287 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc79e4e3-8ded-4c6b-b938-94e6ed6e26f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:50.570379 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc79e4e3-8ded-4c6b-b938-94e6ed6e26f3 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:50.585182 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-cc79e4e3-8ded-4c6b-b938-94e6ed6e26f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:50.585182 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:50.585182 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:50.585182 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:50.585182 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:50.585182 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:50.585182 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:50.585182 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:50.585182 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:50.585182 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:50.585182 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:50.585182 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:50.585182 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:50.585182 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:50.585182 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:50.586085 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:50.586085 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:50.586085 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:50.586085 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:50.586085 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:50.586085 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:50.586085 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:50.586085 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:50.586085 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:50.586085 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:50.586085 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:50.586085 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:50.586085 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:50.586085 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:50.586085 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:50.586085 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:50.586085 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:50.586988 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:50.586988 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:50.586988 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:50.586988 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:50.586988 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 483/940] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:50.587279 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79e4e3-8ded-4c6b-b938-94e6ed6e26f3 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:50.625799 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:50.642839 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b13e9-b7cb-460a-be94-5b84ec19da47 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:50.678389 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0948e3c2-fa34-41ec-b906-a76b1a99ccd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:50.679112 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0948e3c2-fa34-41ec-b906-a76b1a99ccd5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:50.679487 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0948e3c2-fa34-41ec-b906-a76b1a99ccd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:50.679840 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0948e3c2-fa34-41ec-b906-a76b1a99ccd5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:50.687005 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0948e3c2-fa34-41ec-b906-a76b1a99ccd5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:50.688426 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0948e3c2-fa34-41ec-b906-a76b1a99ccd5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:50.694901 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79e4e3-8ded-4c6b-b938-94e6ed6e26f3 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:50.699814 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-0948e3c2-fa34-41ec-b906-a76b1a99ccd5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:50.699814 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:50.699814 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:50.699814 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:50.699814 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:50.699814 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:50.699814 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:50.699814 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:50.699814 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:50.699814 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:50.699814 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:50.699814 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:50.699814 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:56:50.699814 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:56:50.699814 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:56:50.700772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:56:50.700772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:50.700772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:50.700772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:50.700772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:50.700772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:50.700772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:50.700772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:50.700772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:50.700772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:50.700772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:50.700772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:50.700772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:50.700772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:50.700772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:50.700772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:50.700772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:50.705527 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:56:50.705527 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:56:50.705527 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:56:50.705527 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:56:50.705527 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:56:50.705527 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:56:50.705527 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:50.705527 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:50.705527 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 458/941] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:56:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:50.705527 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0948e3c2-fa34-41ec-b906-a76b1a99ccd5 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:50.712944 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:50.736900 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a29131-7ae3-443d-bb4e-4f510a9ce698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:50.737668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a29131-7ae3-443d-bb4e-4f510a9ce698 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:50.738084 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a29131-7ae3-443d-bb4e-4f510a9ce698 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:50.738440 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a29131-7ae3-443d-bb4e-4f510a9ce698 None admin] PATH_INFO: `/v3/services` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:50.739340 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a29131-7ae3-443d-bb4e-4f510a9ce698 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:50.740218 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a29131-7ae3-443d-bb4e-4f510a9ce698 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:50.741691 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a29131-7ae3-443d-bb4e-4f510a9ce698 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:50.754068 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 484/942] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 1 13:56:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:50.754068 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a29131-7ae3-443d-bb4e-4f510a9ce698 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:50.829992 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0948e3c2-fa34-41ec-b906-a76b1a99ccd5 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:50.865701 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43fb034-4e66-4862-ae8a-52a09a83a986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:50.866564 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43fb034-4e66-4862-ae8a-52a09a83a986 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:50.866851 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43fb034-4e66-4862-ae8a-52a09a83a986 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:50.867134 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43fb034-4e66-4862-ae8a-52a09a83a986 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:50.867865 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43fb034-4e66-4862-ae8a-52a09a83a986 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:50.868173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43fb034-4e66-4862-ae8a-52a09a83a986 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:50.869560 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43fb034-4e66-4862-ae8a-52a09a83a986 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:50.870290 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a29131-7ae3-443d-bb4e-4f510a9ce698 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:50.882267 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 459/943] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:50.883173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43fb034-4e66-4862-ae8a-52a09a83a986 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:50.891746 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:50.912432 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6353e2-a4cc-4d29-9597-6738b2563e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:50.913077 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6353e2-a4cc-4d29-9597-6738b2563e21 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:50.913366 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6353e2-a4cc-4d29-9597-6738b2563e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:50.913642 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6353e2-a4cc-4d29-9597-6738b2563e21 None admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:50.914370 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6353e2-a4cc-4d29-9597-6738b2563e21 None admin] RBAC: Adding query filter params (name=service) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:50.914678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6353e2-a4cc-4d29-9597-6738b2563e21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:50.916184 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6353e2-a4cc-4d29-9597-6738b2563e21 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:50.948949 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 485/944] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 13:56:50 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:50.949686 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6353e2-a4cc-4d29-9597-6738b2563e21 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:51.002368 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43fb034-4e66-4862-ae8a-52a09a83a986 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:51.008297 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:51.060948 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b0f947-377f-4b17-a9a9-a122f9dfefff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:51.062613 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0f947-377f-4b17-a9a9-a122f9dfefff None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:51.063204 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0f947-377f-4b17-a9a9-a122f9dfefff None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:51.063667 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0f947-377f-4b17-a9a9-a122f9dfefff None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:51.064863 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b0f947-377f-4b17-a9a9-a122f9dfefff None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:51.065393 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b0f947-377f-4b17-a9a9-a122f9dfefff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:51.067677 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b0f947-377f-4b17-a9a9-a122f9dfefff None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:51.075590 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 460/945] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 1 13:56:50 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:51.081469 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b0f947-377f-4b17-a9a9-a122f9dfefff None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:51.092448 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6353e2-a4cc-4d29-9597-6738b2563e21 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:51.137582 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7882a3-0b82-41ba-b2c1-c5100b2307b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:51.138342 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7882a3-0b82-41ba-b2c1-c5100b2307b6 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:51.138577 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7882a3-0b82-41ba-b2c1-c5100b2307b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:51.138857 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7882a3-0b82-41ba-b2c1-c5100b2307b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:51.148175 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7882a3-0b82-41ba-b2c1-c5100b2307b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:51.149756 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7882a3-0b82-41ba-b2c1-c5100b2307b6 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:51.163245 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-ca7882a3-0b82-41ba-b2c1-c5100b2307b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:51.163245 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:51.163245 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:51.163245 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:51.163245 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:51.163245 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:51.163245 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:51.163245 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:51.163245 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:51.163245 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:51.163245 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:51.163245 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:51.163245 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:56:51.163245 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:56:51.163245 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:56:51.164341 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:56:51.164341 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:51.164341 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:51.164341 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:51.164341 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:51.164341 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:51.164341 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:51.164341 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:51.164341 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:51.164341 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:51.164341 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:51.164341 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:51.164341 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:51.164341 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:51.164341 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:51.164341 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:51.164341 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:51.165546 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:56:51.165546 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:56:51.165546 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:56:51.165546 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:56:51.165546 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:56:51.165546 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:56:51.165546 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:51.165546 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:51.165546 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 486/946] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:56:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:51.171211 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7882a3-0b82-41ba-b2c1-c5100b2307b6 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:51.179461 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:51.224149 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b0f947-377f-4b17-a9a9-a122f9dfefff None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:51.262320 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f78dc-91a7-422e-be0c-81edaa01afb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:51.262953 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f78dc-91a7-422e-be0c-81edaa01afb3 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:51.263238 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f78dc-91a7-422e-be0c-81edaa01afb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:51.263516 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f78dc-91a7-422e-be0c-81edaa01afb3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:51.264229 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52f78dc-91a7-422e-be0c-81edaa01afb3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:51.265621 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52f78dc-91a7-422e-be0c-81edaa01afb3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:51.287966 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:51.296590 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 461/947] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 13:56:51 2022] POST /identity/v3/endpoints => generated 347 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:56:51.298501 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7882a3-0b82-41ba-b2c1-c5100b2307b6 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:51.328292 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78913c5d-34c2-4ccb-8bc0-2179b9f7dd38 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:51.328499 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78913c5d-34c2-4ccb-8bc0-2179b9f7dd38 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:51.328808 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78913c5d-34c2-4ccb-8bc0-2179b9f7dd38 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:51.331523 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 462/948] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:51.337181 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714160c1-3648-41df-b80d-82b04e2e6ed5 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:51.337468 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714160c1-3648-41df-b80d-82b04e2e6ed5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:51.337751 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714160c1-3648-41df-b80d-82b04e2e6ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:51.357803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441130d-b400-4668-863f-59f6ce7569fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:51.358159 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-714160c1-3648-41df-b80d-82b04e2e6ed5 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:51.358432 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441130d-b400-4668-863f-59f6ce7569fa None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:51.358715 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441130d-b400-4668-863f-59f6ce7569fa None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:51.358994 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441130d-b400-4668-863f-59f6ce7569fa None admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:51.359306 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-714160c1-3648-41df-b80d-82b04e2e6ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:51.359826 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d441130d-b400-4668-863f-59f6ce7569fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:51.360059 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d441130d-b400-4668-863f-59f6ce7569fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:51.361866 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d441130d-b400-4668-863f-59f6ce7569fa None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:51.383147 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 487/949] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 13:56:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:51.389444 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441130d-b400-4668-863f-59f6ce7569fa None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:51.456694 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 463/950] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:56:51.480542 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714160c1-3648-41df-b80d-82b04e2e6ed5 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:51.514315 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441130d-b400-4668-863f-59f6ce7569fa None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:51.549230 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:51.565390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c483c2-e265-4230-a609-b9ec726777e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:51.566088 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c483c2-e265-4230-a609-b9ec726777e1 None admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:51.566374 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c483c2-e265-4230-a609-b9ec726777e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:51.566657 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c483c2-e265-4230-a609-b9ec726777e1 None admin] PATH_INFO: `/v3/projects/72ecec9fad964b1fbeaefd0519597229/users/5007a2f8e6244b0788f0a554cc572fe5/roles/05973988fe294b63969752e9dc8da875` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:51.569568 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c483c2-e265-4230-a609-b9ec726777e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=72ecec9fad964b1fbeaefd0519597229, user_id=5007a2f8e6244b0788f0a554cc572fe5, role_id=05973988fe294b63969752e9dc8da875)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:51.571508 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c483c2-e265-4230-a609-b9ec726777e1 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:51.592457 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 488/951] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 13:56:51 2022] PUT /identity/v3/projects/72ecec9fad964b1fbeaefd0519597229/users/5007a2f8e6244b0788f0a554cc572fe5/roles/05973988fe294b63969752e9dc8da875 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 13:56:51.607591 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714160c1-3648-41df-b80d-82b04e2e6ed5 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:51.659209 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa9e08-0fac-4b5a-9605-cf974a6cc820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:51.659934 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa9e08-0fac-4b5a-9605-cf974a6cc820 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:51.660222 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa9e08-0fac-4b5a-9605-cf974a6cc820 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:51.660510 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa9e08-0fac-4b5a-9605-cf974a6cc820 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:51.661775 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aa9e08-0fac-4b5a-9605-cf974a6cc820 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:51.663179 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aa9e08-0fac-4b5a-9605-cf974a6cc820 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:51.673380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-19aa9e08-0fac-4b5a-9605-cf974a6cc820 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 13:56:51.673380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:51.673380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:56:51.673380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:56:51.673380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:56:51.673380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:56:51.673380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:56:51.673380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:51.673380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 13:56:51.673380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:51.673380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:56:51.673380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:51.673380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:51.673380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:51.673380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:51.673380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:51.673380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:51.674435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:51.674435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:51.674435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:51.674435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:51.674435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:51.674435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:51.674435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:56:51.674435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:56:51.674435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:56:51.674435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:56:51.674435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:51.674435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:51.674435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:51.674435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:51.674435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:51.674435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:51.675244 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:51.675244 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:51.675244 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:51.675244 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:51.675244 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:51.675244 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:51.675244 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:51.675244 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:51.675244 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:51.675244 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:51.675244 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:56:51.675244 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:51.675244 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 13:56:51.675244 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:51.680371 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 464/952] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 1 13:56:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:56:51.681771 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c483c2-e265-4230-a609-b9ec726777e1 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:51.705567 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:51.798643 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c483c2-e265-4230-a609-b9ec726777e1 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:51.839065 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3dfb4b-eab9-494f-bcd6-30be6037622b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:51.839700 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3dfb4b-eab9-494f-bcd6-30be6037622b None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:51.839985 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3dfb4b-eab9-494f-bcd6-30be6037622b None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:51.840265 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3dfb4b-eab9-494f-bcd6-30be6037622b None admin] PATH_INFO: `/v3/services` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:51.840997 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3dfb4b-eab9-494f-bcd6-30be6037622b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:51.841306 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3dfb4b-eab9-494f-bcd6-30be6037622b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:51.842817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3dfb4b-eab9-494f-bcd6-30be6037622b None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:51.848881 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 489/953] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 1 13:56:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:51.854409 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa9e08-0fac-4b5a-9605-cf974a6cc820 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:51.961615 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa9e08-0fac-4b5a-9605-cf974a6cc820 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:51.981875 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:51.996845 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf1bd63-38e4-431e-92b8-ff500c289563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:51.997475 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf1bd63-38e4-431e-92b8-ff500c289563 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:51.997762 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf1bd63-38e4-431e-92b8-ff500c289563 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:51.998044 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf1bd63-38e4-431e-92b8-ff500c289563 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:51.998732 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf1bd63-38e4-431e-92b8-ff500c289563 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:51.999041 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf1bd63-38e4-431e-92b8-ff500c289563 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:52.000408 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf1bd63-38e4-431e-92b8-ff500c289563 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:52.006454 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 465/954] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 1 13:56:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:52.011960 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3dfb4b-eab9-494f-bcd6-30be6037622b None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:52.093163 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:52.118493 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3dfb4b-eab9-494f-bcd6-30be6037622b None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:52.154310 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b5841-5b0a-4b74-a02d-d2575866e35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:52.154922 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b5841-5b0a-4b74-a02d-d2575866e35a None admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:52.155206 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b5841-5b0a-4b74-a02d-d2575866e35a None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:52.155488 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b5841-5b0a-4b74-a02d-d2575866e35a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:52.156213 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5b5841-5b0a-4b74-a02d-d2575866e35a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:52.157615 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5b5841-5b0a-4b74-a02d-d2575866e35a None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:52.179379 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 490/955] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 13:56:52 2022] POST /identity/v3/endpoints => generated 349 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:56:53.136456 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c77f7-1f8f-4f6a-9c29-9f18d3960408 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:53.136970 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c77f7-1f8f-4f6a-9c29-9f18d3960408 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:53.136970 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c77f7-1f8f-4f6a-9c29-9f18d3960408 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:53.138621 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 466/956] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:53.144160 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92995b0-18b6-4a33-a0b3-ab7d14cabd28 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:53.144758 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92995b0-18b6-4a33-a0b3-ab7d14cabd28 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:53.145771 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92995b0-18b6-4a33-a0b3-ab7d14cabd28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:53.166415 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-c92995b0-18b6-4a33-a0b3-ab7d14cabd28 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:53.168764 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-c92995b0-18b6-4a33-a0b3-ab7d14cabd28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:53.244867 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 491/957] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:56:53.268994 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c77f7-1f8f-4f6a-9c29-9f18d3960408 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:53.376870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c77f7-1f8f-4f6a-9c29-9f18d3960408 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:53.417106 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240be0b-7d78-4349-9ba6-dc7ead744a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:53.417745 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240be0b-7d78-4349-9ba6-dc7ead744a8a None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:53.418033 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240be0b-7d78-4349-9ba6-dc7ead744a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:53.418318 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240be0b-7d78-4349-9ba6-dc7ead744a8a None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:53.419016 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240be0b-7d78-4349-9ba6-dc7ead744a8a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:53.420446 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240be0b-7d78-4349-9ba6-dc7ead744a8a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:53.423013 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: WARNING keystone.server.flask.common [None req-c240be0b-7d78-4349-9ba6-dc7ead744a8a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 01 13:56:53.437334 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.sql.core [None req-c240be0b-7d78-4349-9ba6-dc7ead744a8a None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 13:56:53.437334 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 01 13:56:53.437334 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [parameters: {'user_id': 'a5b16f5413eb480f91aeb157eeb34738', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 01 13:56:53.437334 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80141) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 13:56:53.455495 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-c240be0b-7d78-4349-9ba6-dc7ead744a8a None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 01 13:56:53.455495 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:53.455495 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 13:56:53.455495 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 13:56:53.455495 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 13:56:53.455495 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 13:56:53.455495 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 13:56:53.455495 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 13:56:53.455495 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 13:56:53.455495 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application conn.query(q) Aug 01 13:56:53.455495 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 13:56:53.455495 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 13:56:53.455495 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 13:56:53.455495 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application result.read() Aug 01 13:56:53.455495 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 13:56:53.456427 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 13:56:53.456427 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 13:56:53.456427 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 13:56:53.456427 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 13:56:53.456427 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 13:56:53.456427 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 13:56:53.456427 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 13:56:53.456427 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 13:56:53.456427 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:53.456427 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 13:56:53.456427 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:53.456427 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:53.456427 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 13:56:53.456427 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 13:56:53.456427 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 01 13:56:53.456427 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 01 13:56:53.456427 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 13:56:53.456427 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application next(self.gen) Aug 01 13:56:53.457417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 13:56:53.457417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application yield resource Aug 01 13:56:53.457417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 13:56:53.457417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application next(self.gen) Aug 01 13:56:53.457417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 13:56:53.457417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 13:56:53.457417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 13:56:53.457417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 13:56:53.457417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 13:56:53.457417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise self.value Aug 01 13:56:53.457417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 13:56:53.457417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 13:56:53.457417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 13:56:53.457417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application session.commit() Aug 01 13:56:53.457417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 13:56:53.457417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 13:56:53.457417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 13:56:53.458892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 13:56:53.458892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 13:56:53.458892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self.session.flush() Aug 01 13:56:53.458892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 13:56:53.458892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 13:56:53.458892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 13:56:53.458892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 13:56:53.458892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 13:56:53.458892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application compat.raise_( Aug 01 13:56:53.458892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 13:56:53.458892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception Aug 01 13:56:53.458892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 13:56:53.458892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 13:56:53.458892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 13:56:53.458892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 13:56:53.458892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 13:56:53.458892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 13:56:53.461179 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 13:56:53.461179 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 13:56:53.461179 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 01 13:56:53.461179 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 01 13:56:53.461179 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 13:56:53.461179 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 13:56:53.461179 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 13:56:53.461179 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 13:56:53.461179 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 13:56:53.461179 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 13:56:53.461179 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 13:56:53.461179 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 13:56:53.461179 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 13:56:53.461179 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 13:56:53.461179 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 13:56:53.461179 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception Aug 01 13:56:53.463155 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 13:56:53.463155 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 13:56:53.463155 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 13:56:53.463155 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 13:56:53.463155 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 13:56:53.463155 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 13:56:53.463155 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 13:56:53.463155 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application conn.query(q) Aug 01 13:56:53.463155 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 13:56:53.463155 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 13:56:53.463155 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 13:56:53.463155 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application result.read() Aug 01 13:56:53.463155 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 13:56:53.463155 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 13:56:53.463155 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 13:56:53.463155 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 13:56:53.465224 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 13:56:53.465224 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 13:56:53.465224 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 13:56:53.465224 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 13:56:53.465224 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 13:56:53.465224 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 01 13:56:53.465224 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [parameters: {'user_id': 'a5b16f5413eb480f91aeb157eeb34738', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 01 13:56:53.465224 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 13:56:53.465224 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:53.465224 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:56:53.465224 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:53.465224 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:53.465224 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:53.465224 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:53.465224 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:53.472772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:53.472772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:53.472772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:53.472772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:53.472772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:53.472772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:53.472772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:53.472772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 01 13:56:53.472772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 01 13:56:53.472772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:53.472772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:53.472772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:56:53.472772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:53.472772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:56:53.472772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:53.472772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 01 13:56:53.472772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 01 13:56:53.474965 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 01 13:56:53.474965 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 01 13:56:53.474965 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 13:56:53.474965 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 13:56:53.474965 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 01 13:56:53.474965 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:53.474965 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 467/958] 10.222.0.48 () {60 vars in 1131 bytes} [Mon Aug 1 13:56:53 2022] POST /identity/v3/users => generated 160 bytes in 189 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:53.477107 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92995b0-18b6-4a33-a0b3-ab7d14cabd28 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:53.477899 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21abda-1403-45c3-a23a-5e6e77a70e64 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:53.478193 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21abda-1403-45c3-a23a-5e6e77a70e64 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:53.478474 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21abda-1403-45c3-a23a-5e6e77a70e64 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:53.486493 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 468/959] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:56:53.487968 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163174b-457e-4bdf-864a-a61fe39bd5c0 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:53.488237 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163174b-457e-4bdf-864a-a61fe39bd5c0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:53.488551 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163174b-457e-4bdf-864a-a61fe39bd5c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:53.509926 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4163174b-457e-4bdf-864a-a61fe39bd5c0 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:53.511817 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4163174b-457e-4bdf-864a-a61fe39bd5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:53.552176 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:53.584840 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:53.596331 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 469/960] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:56:53.606735 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92995b0-18b6-4a33-a0b3-ab7d14cabd28 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:53.619174 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163174b-457e-4bdf-864a-a61fe39bd5c0 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:53.643959 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a21f5-a9b0-420c-8a8e-9d46f2d445eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:53.645012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a21f5-a9b0-420c-8a8e-9d46f2d445eb None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:53.645499 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a21f5-a9b0-420c-8a8e-9d46f2d445eb None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:53.645989 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a21f5-a9b0-420c-8a8e-9d46f2d445eb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:53.653693 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a21f5-a9b0-420c-8a8e-9d46f2d445eb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:53.655394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a21f5-a9b0-420c-8a8e-9d46f2d445eb None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:53.676927 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-910a21f5-a9b0-420c-8a8e-9d46f2d445eb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 13:56:53.676927 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:53.676927 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:53.676927 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:53.676927 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:53.676927 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:53.676927 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:53.676927 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:53.676927 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:53.676927 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:53.676927 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:53.676927 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:53.676927 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:56:53.676927 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:56:53.676927 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:56:53.676927 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:56:53.677895 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:53.677895 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:53.677895 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:56:53.677895 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:53.677895 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:56:53.677895 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:53.677895 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:53.677895 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:53.677895 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:53.677895 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:53.677895 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:53.677895 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:53.677895 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:53.677895 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:53.677895 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:53.677895 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:53.677895 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:53.677895 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:53.678878 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:53.678878 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:53.678878 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:56:53.678878 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:56:53.678878 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:56:53.678878 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:56:53.678878 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:56:53.678878 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:56:53.678878 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 13:56:53.678878 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:53.680549 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 492/961] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:56:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:53.685038 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a21f5-a9b0-420c-8a8e-9d46f2d445eb None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:53.746015 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163174b-457e-4bdf-864a-a61fe39bd5c0 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:53.788000 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c3308-13fe-4f37-b513-88ead6dea496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:53.789182 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c3308-13fe-4f37-b513-88ead6dea496 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:53.789467 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c3308-13fe-4f37-b513-88ead6dea496 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:53.789880 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c3308-13fe-4f37-b513-88ead6dea496 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:53.790637 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c3308-13fe-4f37-b513-88ead6dea496 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:53.792961 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c3308-13fe-4f37-b513-88ead6dea496 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:53.804340 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a21f5-a9b0-420c-8a8e-9d46f2d445eb None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:53.807977 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-381c3308-13fe-4f37-b513-88ead6dea496 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 13:56:53.807977 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:53.807977 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:56:53.807977 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:56:53.807977 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:56:53.807977 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:56:53.807977 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:56:53.807977 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:53.807977 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 13:56:53.807977 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:53.807977 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:56:53.807977 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:53.807977 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:53.807977 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:53.807977 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:53.807977 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:53.807977 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:53.809045 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:53.809045 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:53.809045 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:53.809045 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:53.809045 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:53.809045 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:53.809045 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:56:53.809045 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:56:53.809045 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:56:53.809045 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:56:53.809045 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:53.809045 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:53.809045 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:53.809045 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:53.809045 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:53.809045 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:53.809908 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:53.809908 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:53.809908 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:53.809908 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:53.809908 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:53.809908 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:53.809908 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:53.809908 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:53.809908 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:53.809908 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:53.809908 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:56:53.809908 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:53.809908 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 13:56:53.809908 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:53.809908 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 470/962] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:56:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:56:53.810707 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1261a-fb30-40d0-967c-fe2288d5d3c3 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:53.811276 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1261a-fb30-40d0-967c-fe2288d5d3c3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:53.811276 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1261a-fb30-40d0-967c-fe2288d5d3c3 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:53.812675 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 471/963] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:53.820325 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda7b78-4b46-4125-ae09-691a8c296fe2 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:53.820325 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda7b78-4b46-4125-ae09-691a8c296fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:53.820325 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda7b78-4b46-4125-ae09-691a8c296fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:53.829181 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:53.837182 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-bfda7b78-4b46-4125-ae09-691a8c296fe2 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:53.838390 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-bfda7b78-4b46-4125-ae09-691a8c296fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:53.847940 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a072ad-f68c-4620-8a35-09390bb3a13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:53.849365 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a072ad-f68c-4620-8a35-09390bb3a13f None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:53.849510 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a072ad-f68c-4620-8a35-09390bb3a13f None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:53.849802 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a072ad-f68c-4620-8a35-09390bb3a13f None admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:53.850572 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a072ad-f68c-4620-8a35-09390bb3a13f None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:53.850886 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a072ad-f68c-4620-8a35-09390bb3a13f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:53.852359 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a072ad-f68c-4620-8a35-09390bb3a13f None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:53.875843 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:53.896519 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 493/964] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 13:56:53 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:53.896519 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a072ad-f68c-4620-8a35-09390bb3a13f None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:53.970163 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 472/965] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:56:54.014076 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a072ad-f68c-4620-8a35-09390bb3a13f None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:54.016675 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfda7b78-4b46-4125-ae09-691a8c296fe2 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:54.056091 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f0cba-0e65-4af0-9a24-8555f990cd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:54.057478 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f0cba-0e65-4af0-9a24-8555f990cd14 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:54.057982 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f0cba-0e65-4af0-9a24-8555f990cd14 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:54.058453 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f0cba-0e65-4af0-9a24-8555f990cd14 None admin] PATH_INFO: `/v3/services` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:54.059344 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f0cba-0e65-4af0-9a24-8555f990cd14 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:54.059910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f0cba-0e65-4af0-9a24-8555f990cd14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:54.061609 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f0cba-0e65-4af0-9a24-8555f990cd14 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:54.068087 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 494/966] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 13:56:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:54.077773 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f0cba-0e65-4af0-9a24-8555f990cd14 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:54.135403 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:54.227767 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfda7b78-4b46-4125-ae09-691a8c296fe2 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:54.236318 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f0cba-0e65-4af0-9a24-8555f990cd14 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:54.273965 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb22ce7a-b5f3-4eb1-af43-e9dfb72e69f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:54.274987 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22ce7a-b5f3-4eb1-af43-e9dfb72e69f1 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:54.275474 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22ce7a-b5f3-4eb1-af43-e9dfb72e69f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:54.275959 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22ce7a-b5f3-4eb1-af43-e9dfb72e69f1 None admin] PATH_INFO: `/v3/services` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:54.276912 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb22ce7a-b5f3-4eb1-af43-e9dfb72e69f1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:54.277422 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb22ce7a-b5f3-4eb1-af43-e9dfb72e69f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:54.279003 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb22ce7a-b5f3-4eb1-af43-e9dfb72e69f1 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:54.280202 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f02ce6-b491-4d40-a292-9b28e1c45d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:54.280979 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f02ce6-b491-4d40-a292-9b28e1c45d1c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:54.281275 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f02ce6-b491-4d40-a292-9b28e1c45d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:54.281542 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f02ce6-b491-4d40-a292-9b28e1c45d1c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:54.293000 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 495/967] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 13:56:54 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:56:54.293559 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f02ce6-b491-4d40-a292-9b28e1c45d1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:56:54.298982 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-90f02ce6-b491-4d40-a292-9b28e1c45d1c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 13:56:54.298982 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:54.298982 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:54.298982 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:54.298982 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:54.298982 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:54.298982 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:54.298982 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:54.298982 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:54.298982 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:54.298982 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:54.298982 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:54.298982 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:56:54.298982 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:56:54.298982 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:56:54.298982 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:56:54.299989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:56:54.299989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:56:54.299989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:56:54.299989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:56:54.299989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:54.299989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:54.299989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:54.299989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:54.299989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:54.299989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:54.299989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:54.299989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:54.299989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:54.299989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:54.299989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:54.299989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:54.300861 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:54.300861 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:54.300861 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:54.300861 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:54.300861 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:56:54.300861 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:56:54.300861 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:56:54.300861 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:56:54.300861 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:56:54.300861 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:56:54.300861 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 13:56:54.300861 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:54.301760 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75b6ff-faae-420a-9362-432b9ec6c009 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:54.301859 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 473/968] 10.222.0.48 () {56 vars in 1120 bytes} [Mon Aug 1 13:56:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:54.301944 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75b6ff-faae-420a-9362-432b9ec6c009 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:54.302033 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75b6ff-faae-420a-9362-432b9ec6c009 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:54.308525 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 496/969] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:56:54.308753 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f02ce6-b491-4d40-a292-9b28e1c45d1c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:54.309096 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d8459-8ebc-401a-8f69-168ec3e581eb None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:54.309390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d8459-8ebc-401a-8f69-168ec3e581eb None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:54.309668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d8459-8ebc-401a-8f69-168ec3e581eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:54.335889 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-9a3d8459-8ebc-401a-8f69-168ec3e581eb None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:54.338938 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-9a3d8459-8ebc-401a-8f69-168ec3e581eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:54.354961 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:54.426429 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 497/970] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:56:54.449105 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d8459-8ebc-401a-8f69-168ec3e581eb None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:54.451568 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f02ce6-b491-4d40-a292-9b28e1c45d1c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:54.492533 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2379ff9-fa99-4c0b-bd8a-0ba255dc0dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:54.493086 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2379ff9-fa99-4c0b-bd8a-0ba255dc0dfa None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:54.493365 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2379ff9-fa99-4c0b-bd8a-0ba255dc0dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:54.493641 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2379ff9-fa99-4c0b-bd8a-0ba255dc0dfa None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:54.494357 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2379ff9-fa99-4c0b-bd8a-0ba255dc0dfa None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:54.494661 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2379ff9-fa99-4c0b-bd8a-0ba255dc0dfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:54.495577 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:54.496086 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2379ff9-fa99-4c0b-bd8a-0ba255dc0dfa None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:54.505565 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 474/971] 10.222.0.48 () {56 vars in 1115 bytes} [Mon Aug 1 13:56:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:54.510823 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2379ff9-fa99-4c0b-bd8a-0ba255dc0dfa None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:54.572364 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d8459-8ebc-401a-8f69-168ec3e581eb None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:54.607906 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b24945-7754-4baa-b71e-172215b1f242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:54.608585 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b24945-7754-4baa-b71e-172215b1f242 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:54.608871 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b24945-7754-4baa-b71e-172215b1f242 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:54.609148 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b24945-7754-4baa-b71e-172215b1f242 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:54.609782 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b24945-7754-4baa-b71e-172215b1f242 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:54.611152 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b24945-7754-4baa-b71e-172215b1f242 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:54.616339 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:54.620305 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-a5b24945-7754-4baa-b71e-172215b1f242 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 13:56:54.620305 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:54.620305 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:56:54.620305 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:56:54.620305 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:56:54.620305 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:56:54.620305 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:56:54.620305 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:54.620305 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 13:56:54.620305 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:54.620305 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:56:54.620305 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:54.620305 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:54.620305 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:54.620305 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:54.620305 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:54.620305 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:54.621422 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:54.621422 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:54.621422 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:54.621422 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:54.621422 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:54.621422 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:54.621422 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:56:54.621422 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:56:54.621422 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:56:54.621422 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:56:54.621422 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:54.621422 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:54.621422 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:54.621422 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:54.621422 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:54.621422 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:54.622242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:54.622242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:54.622242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:54.622242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:54.622242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:54.622242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:54.622242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:54.622242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:54.622242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:54.622242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:54.622242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:56:54.622242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:54.622242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 13:56:54.622242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:54.622242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 498/972] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 1 13:56:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:54.627474 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b24945-7754-4baa-b71e-172215b1f242 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:54.630082 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2379ff9-fa99-4c0b-bd8a-0ba255dc0dfa None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:54.667168 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e01eb5-118c-40aa-8663-c226350fe281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:54.667814 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e01eb5-118c-40aa-8663-c226350fe281 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:54.668098 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e01eb5-118c-40aa-8663-c226350fe281 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:54.668373 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e01eb5-118c-40aa-8663-c226350fe281 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:54.679982 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e01eb5-118c-40aa-8663-c226350fe281 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:54.681452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e01eb5-118c-40aa-8663-c226350fe281 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:54.696743 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-15e01eb5-118c-40aa-8663-c226350fe281 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:54.696743 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:54.696743 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:54.696743 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:54.696743 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:54.696743 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:54.696743 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:54.696743 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:54.696743 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:54.696743 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:54.696743 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:54.696743 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:54.696743 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:54.696743 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:54.696743 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:54.697644 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:54.697644 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:54.697644 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:54.697644 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:54.697644 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:54.697644 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:54.697644 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:54.697644 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:54.697644 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:54.697644 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:54.697644 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:54.697644 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:54.697644 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:54.697644 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:54.697644 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:54.697644 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:54.697644 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:54.698542 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:54.698542 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:54.698542 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:54.698542 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:54.702583 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 475/973] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:56:54.703425 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e01eb5-118c-40aa-8663-c226350fe281 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:54.731473 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:54.737306 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b24945-7754-4baa-b71e-172215b1f242 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:54.776350 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680bd1c2-6554-4baf-bd24-2e9c8e6e831b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:54.776924 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680bd1c2-6554-4baf-bd24-2e9c8e6e831b None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:54.777240 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680bd1c2-6554-4baf-bd24-2e9c8e6e831b None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:54.777482 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680bd1c2-6554-4baf-bd24-2e9c8e6e831b None admin] PATH_INFO: `/v3/services` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:54.778164 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680bd1c2-6554-4baf-bd24-2e9c8e6e831b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:54.778527 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680bd1c2-6554-4baf-bd24-2e9c8e6e831b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:54.780177 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680bd1c2-6554-4baf-bd24-2e9c8e6e831b None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:54.789073 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 499/974] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 1 13:56:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:54.790086 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680bd1c2-6554-4baf-bd24-2e9c8e6e831b None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:54.826692 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e01eb5-118c-40aa-8663-c226350fe281 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:54.875359 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b17d5-a248-48b6-a009-7706b053b684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:54.876270 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b17d5-a248-48b6-a009-7706b053b684 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:54.876735 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b17d5-a248-48b6-a009-7706b053b684 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:54.877161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b17d5-a248-48b6-a009-7706b053b684 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:54.878245 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b17d5-a248-48b6-a009-7706b053b684 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:54.878722 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b17d5-a248-48b6-a009-7706b053b684 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:54.881733 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b17d5-a248-48b6-a009-7706b053b684 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:54.900085 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 476/975] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:54.906554 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b17d5-a248-48b6-a009-7706b053b684 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:54.911146 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680bd1c2-6554-4baf-bd24-2e9c8e6e831b None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:54.911703 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:54.949968 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7619bec-5863-4215-9026-488c1b2d4ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:54.950594 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7619bec-5863-4215-9026-488c1b2d4ed5 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:54.950880 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7619bec-5863-4215-9026-488c1b2d4ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:54.951158 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7619bec-5863-4215-9026-488c1b2d4ed5 None admin] PATH_INFO: `/v3/services` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:54.951856 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7619bec-5863-4215-9026-488c1b2d4ed5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:54.952154 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7619bec-5863-4215-9026-488c1b2d4ed5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:54.953550 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7619bec-5863-4215-9026-488c1b2d4ed5 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:54.958597 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 500/976] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 1 13:56:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:56:54.964848 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7619bec-5863-4215-9026-488c1b2d4ed5 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:54.993678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:55.038153 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b17d5-a248-48b6-a009-7706b053b684 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:55.074580 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7619bec-5863-4215-9026-488c1b2d4ed5 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:55.077806 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61d0a4-e8c4-45e5-a448-6d66bc7cdbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:55.078418 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61d0a4-e8c4-45e5-a448-6d66bc7cdbb3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:55.078701 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61d0a4-e8c4-45e5-a448-6d66bc7cdbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:55.078975 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61d0a4-e8c4-45e5-a448-6d66bc7cdbb3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:55.086206 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e61d0a4-e8c4-45e5-a448-6d66bc7cdbb3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:55.087642 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e61d0a4-e8c4-45e5-a448-6d66bc7cdbb3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:55.100473 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-1e61d0a4-e8c4-45e5-a448-6d66bc7cdbb3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 13:56:55.100473 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:55.100473 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:55.100473 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:55.100473 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:55.100473 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:55.100473 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:55.100473 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:55.100473 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:55.100473 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:55.100473 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:55.100473 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:55.100473 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:56:55.100473 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:56:55.100473 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:56:55.100473 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:56:55.101452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:55.101452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:55.101452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:56:55.101452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:55.101452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:56:55.101452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:56:55.101452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:55.101452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:55.101452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:55.101452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:55.101452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:55.101452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:55.101452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:55.101452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:55.101452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:55.101452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:55.101452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:55.101452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:55.102375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:55.102375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:55.102375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:56:55.102375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:56:55.102375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:56:55.102375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:56:55.102375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:56:55.102375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:56:55.102375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 13:56:55.102375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:55.102375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 477/977] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 1 13:56:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:55.107406 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61d0a4-e8c4-45e5-a448-6d66bc7cdbb3 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:55.112512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56054853-7f0d-4136-a824-4ba533c3e191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:55.113129 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56054853-7f0d-4136-a824-4ba533c3e191 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:55.113417 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56054853-7f0d-4136-a824-4ba533c3e191 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:55.113746 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56054853-7f0d-4136-a824-4ba533c3e191 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:55.115020 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56054853-7f0d-4136-a824-4ba533c3e191 None admin] RBAC: Adding query filter params (interface=admin, service_id=cf76751b2bcf45b1a6206435262ceddd, region_id=RegionOne) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:55.115334 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56054853-7f0d-4136-a824-4ba533c3e191 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:55.116722 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56054853-7f0d-4136-a824-4ba533c3e191 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:55.127022 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 501/978] 10.222.0.48 () {56 vars in 1249 bytes} [Mon Aug 1 13:56:54 2022] GET /identity/v3/endpoints?service_id=cf76751b2bcf45b1a6206435262ceddd&interface=admin®ion_id=RegionOne => generated 193 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:56:55.127826 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56054853-7f0d-4136-a824-4ba533c3e191 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:55.177578 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:55.218144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61d0a4-e8c4-45e5-a448-6d66bc7cdbb3 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:55.234473 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:55.251237 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56054853-7f0d-4136-a824-4ba533c3e191 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:55.266634 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cfc6fb-2367-4d17-b4e4-5079b2516ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:55.267270 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cfc6fb-2367-4d17-b4e4-5079b2516ece None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:55.267561 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cfc6fb-2367-4d17-b4e4-5079b2516ece None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:55.267850 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cfc6fb-2367-4d17-b4e4-5079b2516ece None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:55.268901 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cfc6fb-2367-4d17-b4e4-5079b2516ece None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:55.269407 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cfc6fb-2367-4d17-b4e4-5079b2516ece None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:55.271791 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cfc6fb-2367-4d17-b4e4-5079b2516ece None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:55.291406 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ddd91-cd3c-4d6c-8c47-4f852b3a9db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:55.291962 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ddd91-cd3c-4d6c-8c47-4f852b3a9db3 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:55.296705 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ddd91-cd3c-4d6c-8c47-4f852b3a9db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:55.296705 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ddd91-cd3c-4d6c-8c47-4f852b3a9db3 None admin] PATH_INFO: `/v3/services` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:55.296926 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 478/979] 10.222.0.48 () {56 vars in 1149 bytes} [Mon Aug 1 13:56:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:56:55.297015 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9ddd91-cd3c-4d6c-8c47-4f852b3a9db3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:55.297015 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9ddd91-cd3c-4d6c-8c47-4f852b3a9db3 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:55.297712 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cfc6fb-2367-4d17-b4e4-5079b2516ece None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:55.303353 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 502/980] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Aug 1 13:56:55 2022] GET /identity/v3/services => generated 1497 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 13:56:55.419875 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cfc6fb-2367-4d17-b4e4-5079b2516ece None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:55.445406 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:55.460343 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146e11b-aa06-48a0-8cb8-a5f17c6e061e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:55.461001 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146e11b-aa06-48a0-8cb8-a5f17c6e061e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:55.461285 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146e11b-aa06-48a0-8cb8-a5f17c6e061e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:55.461559 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146e11b-aa06-48a0-8cb8-a5f17c6e061e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:55.530435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146e11b-aa06-48a0-8cb8-a5f17c6e061e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:55.531895 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146e11b-aa06-48a0-8cb8-a5f17c6e061e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:55.628022 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 479/981] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:56:55 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 13:56:55.633635 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ddd91-cd3c-4d6c-8c47-4f852b3a9db3 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:55.735521 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ddd91-cd3c-4d6c-8c47-4f852b3a9db3 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:55.770490 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff1d2b-4c38-4b42-a27d-88cde61fb360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:55.771138 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff1d2b-4c38-4b42-a27d-88cde61fb360 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:55.771421 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff1d2b-4c38-4b42-a27d-88cde61fb360 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:55.771695 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff1d2b-4c38-4b42-a27d-88cde61fb360 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:55.784116 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ff1d2b-4c38-4b42-a27d-88cde61fb360 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:55.786116 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ff1d2b-4c38-4b42-a27d-88cde61fb360 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:55.802575 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-67ff1d2b-4c38-4b42-a27d-88cde61fb360 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:55.802575 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:55.802575 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:55.802575 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:55.802575 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:55.802575 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:55.802575 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:55.802575 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:55.802575 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:55.802575 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:55.802575 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:55.802575 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:55.802575 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:56:55.802575 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:56:55.802575 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:56:55.803867 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:56:55.803867 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:55.803867 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:55.803867 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:55.803867 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:55.803867 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:55.803867 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:55.803867 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:55.803867 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:55.803867 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:55.803867 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:55.803867 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:55.803867 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:55.803867 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:55.803867 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:55.803867 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:55.803867 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:55.804828 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:56:55.804828 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:56:55.804828 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:56:55.804828 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:55.807365 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 503/982] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:55.808634 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146e11b-aa06-48a0-8cb8-a5f17c6e061e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:55.836778 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:55.916025 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146e11b-aa06-48a0-8cb8-a5f17c6e061e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:55.951198 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7092de-95d1-4184-a1e9-8aa3f67c07c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:55.951823 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7092de-95d1-4184-a1e9-8aa3f67c07c0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:55.952108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7092de-95d1-4184-a1e9-8aa3f67c07c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:55.952384 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7092de-95d1-4184-a1e9-8aa3f67c07c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:55.953114 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7092de-95d1-4184-a1e9-8aa3f67c07c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:55.953428 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7092de-95d1-4184-a1e9-8aa3f67c07c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:55.954905 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7092de-95d1-4184-a1e9-8aa3f67c07c0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:55.970968 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 480/983] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:56:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:55.972461 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff1d2b-4c38-4b42-a27d-88cde61fb360 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:55.982522 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:56.081031 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff1d2b-4c38-4b42-a27d-88cde61fb360 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:56.115176 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cfdbd4-32c7-48a9-ba1e-2abb1386a19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:56.115941 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cfdbd4-32c7-48a9-ba1e-2abb1386a19c None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:56.116459 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cfdbd4-32c7-48a9-ba1e-2abb1386a19c None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:56.116940 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cfdbd4-32c7-48a9-ba1e-2abb1386a19c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:56.123699 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cfdbd4-32c7-48a9-ba1e-2abb1386a19c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:56.125255 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cfdbd4-32c7-48a9-ba1e-2abb1386a19c None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:56.136774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-f0cfdbd4-32c7-48a9-ba1e-2abb1386a19c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:56.136774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:56.136774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:56.136774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:56.136774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:56.136774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:56.136774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:56.136774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:56.136774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:56.136774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:56.136774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:56.136774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:56.136774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:56:56.136774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:56:56.136774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:56:56.137817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:56:56.137817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:56.137817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:56.137817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:56.137817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:56.137817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:56.137817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:56.137817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:56.137817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:56.137817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:56.137817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:56.137817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:56.137817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:56.137817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:56.137817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:56.137817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:56.137817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:56.138949 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:56:56.138949 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:56:56.138949 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:56:56.138949 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:56:56.138949 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:56:56.138949 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:56:56.138949 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:56:56.138949 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:56.140626 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 504/984] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:56:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:56:56.147075 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7092de-95d1-4184-a1e9-8aa3f67c07c0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:56.231713 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:56.250668 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7092de-95d1-4184-a1e9-8aa3f67c07c0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:56.284281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a870f60-f0cf-4f6d-9390-8d58ffbd2f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:56.284943 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a870f60-f0cf-4f6d-9390-8d58ffbd2f1a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:56.285202 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a870f60-f0cf-4f6d-9390-8d58ffbd2f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:56.286019 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a870f60-f0cf-4f6d-9390-8d58ffbd2f1a None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:56.286779 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a870f60-f0cf-4f6d-9390-8d58ffbd2f1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:56.287089 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a870f60-f0cf-4f6d-9390-8d58ffbd2f1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:56.288520 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a870f60-f0cf-4f6d-9390-8d58ffbd2f1a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:56.300217 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108a558-b4ea-458c-b2e5-9d1f2e9c1519 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:56.300721 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108a558-b4ea-458c-b2e5-9d1f2e9c1519 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:56.300799 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108a558-b4ea-458c-b2e5-9d1f2e9c1519 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:56.302293 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 505/985] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:56.304772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 481/986] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 13:56:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:56.306717 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2108a558-b4ea-458c-b2e5-9d1f2e9c1519 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:56.307560 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f18dc-4c1a-40f6-8042-59388828ab2a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:56.307994 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f18dc-4c1a-40f6-8042-59388828ab2a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:56.308437 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f18dc-4c1a-40f6-8042-59388828ab2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:56.326805 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-438f18dc-4c1a-40f6-8042-59388828ab2a None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:56.328181 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-438f18dc-4c1a-40f6-8042-59388828ab2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:56.381251 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:56.396214 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 482/987] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:56:56.418513 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438f18dc-4c1a-40f6-8042-59388828ab2a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:56.422151 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2108a558-b4ea-458c-b2e5-9d1f2e9c1519 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:56.462030 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb43d613-4f3e-4cf1-beb2-fcb0aaed0afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:56.462656 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb43d613-4f3e-4cf1-beb2-fcb0aaed0afe None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:56.462934 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb43d613-4f3e-4cf1-beb2-fcb0aaed0afe None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:56.463206 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb43d613-4f3e-4cf1-beb2-fcb0aaed0afe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:56.464018 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb43d613-4f3e-4cf1-beb2-fcb0aaed0afe None admin] RBAC: Adding query filter params (role.id=05973988fe294b63969752e9dc8da875, scope.project.id=72ecec9fad964b1fbeaefd0519597229, user.id=5007a2f8e6244b0788f0a554cc572fe5) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:56.464328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb43d613-4f3e-4cf1-beb2-fcb0aaed0afe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:56.465766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb43d613-4f3e-4cf1-beb2-fcb0aaed0afe None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:56.471986 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 506/988] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:56:56 2022] GET /identity/v3/role_assignments?user.id=5007a2f8e6244b0788f0a554cc572fe5&scope.project.id=72ecec9fad964b1fbeaefd0519597229&role.id=05973988fe294b63969752e9dc8da875 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:56.497169 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:56.531050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438f18dc-4c1a-40f6-8042-59388828ab2a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:56.570598 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffded3b-464f-4028-8105-bbefc4c27f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:56.571303 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffded3b-464f-4028-8105-bbefc4c27f55 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:56.571718 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffded3b-464f-4028-8105-bbefc4c27f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:56.572120 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffded3b-464f-4028-8105-bbefc4c27f55 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:56.572861 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffded3b-464f-4028-8105-bbefc4c27f55 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:56.574274 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffded3b-464f-4028-8105-bbefc4c27f55 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:56.591189 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 483/989] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 13:56:56 2022] POST /identity/v3/services => generated 242 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 13:56:57.450223 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6e5c7-22f1-4227-8bd1-dd095750217d None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:57.450776 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6e5c7-22f1-4227-8bd1-dd095750217d None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:57.450776 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6e5c7-22f1-4227-8bd1-dd095750217d None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:57.452177 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 507/990] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:57.457387 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4a679-7785-4147-90a5-e6c137c1ddee None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:57.457781 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4a679-7785-4147-90a5-e6c137c1ddee None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:57.457953 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4a679-7785-4147-90a5-e6c137c1ddee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:57.476393 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c3b4a679-7785-4147-90a5-e6c137c1ddee None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:57.477638 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c3b4a679-7785-4147-90a5-e6c137c1ddee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:57.552532 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 484/991] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:57 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:56:57.573921 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d6e5c7-22f1-4227-8bd1-dd095750217d None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:57.680620 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d6e5c7-22f1-4227-8bd1-dd095750217d None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:57.718839 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:57.736540 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91397f7-5686-436c-9a05-c2a78e7ce41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:57.736540 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91397f7-5686-436c-9a05-c2a78e7ce41b None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:57.736540 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91397f7-5686-436c-9a05-c2a78e7ce41b None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:57.736540 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91397f7-5686-436c-9a05-c2a78e7ce41b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:57.737014 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91397f7-5686-436c-9a05-c2a78e7ce41b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:57.737014 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91397f7-5686-436c-9a05-c2a78e7ce41b None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:57.749857 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-d91397f7-5686-436c-9a05-c2a78e7ce41b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 13:56:57.749857 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:57.749857 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:56:57.749857 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:56:57.749857 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:56:57.749857 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:56:57.749857 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:56:57.749857 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:57.749857 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 13:56:57.749857 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:57.749857 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:56:57.749857 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:57.749857 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:57.749857 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:57.749857 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:57.749857 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:57.749857 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:57.750911 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:57.750911 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:57.750911 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:57.750911 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:57.750911 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:57.750911 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:57.750911 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:56:57.750911 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:56:57.750911 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:56:57.750911 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:56:57.750911 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:57.750911 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:57.750911 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:57.750911 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:57.750911 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:57.750911 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:57.757212 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:57.757212 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:57.757212 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:57.757212 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:57.757212 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:57.757212 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:57.757212 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:57.757212 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:57.757212 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:57.757212 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:57.757212 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:56:57.757212 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:57.757212 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 13:56:57.757212 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:57.757212 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 508/992] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 1 13:56:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:56:57.758011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b4a679-7785-4147-90a5-e6c137c1ddee None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:57.819286 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:57.864365 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b4a679-7785-4147-90a5-e6c137c1ddee None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:57.898491 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa2b4dc-c087-4d03-83a2-ce0ac1190e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:57.899104 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2b4dc-c087-4d03-83a2-ce0ac1190e4a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:57.899383 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2b4dc-c087-4d03-83a2-ce0ac1190e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:57.899654 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2b4dc-c087-4d03-83a2-ce0ac1190e4a None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:57.900320 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa2b4dc-c087-4d03-83a2-ce0ac1190e4a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:57.900664 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa2b4dc-c087-4d03-83a2-ce0ac1190e4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:57.902007 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa2b4dc-c087-4d03-83a2-ce0ac1190e4a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:57.906972 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 485/993] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 1 13:56:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:57.913083 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91397f7-5686-436c-9a05-c2a78e7ce41b None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:58.011096 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91397f7-5686-436c-9a05-c2a78e7ce41b None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:58.045472 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df537a83-873c-4284-afef-9c6a0aee8393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:58.046093 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df537a83-873c-4284-afef-9c6a0aee8393 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:58.046374 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df537a83-873c-4284-afef-9c6a0aee8393 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:58.046647 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df537a83-873c-4284-afef-9c6a0aee8393 None admin] PATH_INFO: `/v3/services` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:58.047322 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df537a83-873c-4284-afef-9c6a0aee8393 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:58.047623 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df537a83-873c-4284-afef-9c6a0aee8393 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:58.049020 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df537a83-873c-4284-afef-9c6a0aee8393 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:58.058689 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 509/994] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 1 13:56:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:58.059955 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa2b4dc-c087-4d03-83a2-ce0ac1190e4a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:58.162016 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa2b4dc-c087-4d03-83a2-ce0ac1190e4a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:58.184863 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:58.197896 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d1a1e-4c36-47e6-a355-d4d67e2d0ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:58.198528 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d1a1e-4c36-47e6-a355-d4d67e2d0ae1 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:58.198831 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d1a1e-4c36-47e6-a355-d4d67e2d0ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:58.199098 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d1a1e-4c36-47e6-a355-d4d67e2d0ae1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:58.199816 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d1a1e-4c36-47e6-a355-d4d67e2d0ae1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:58.201201 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d1a1e-4c36-47e6-a355-d4d67e2d0ae1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:58.218045 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 486/995] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 13:56:58 2022] POST /identity/v3/endpoints => generated 325 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:56:58.496514 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10933e4d-2a8b-47ae-ac23-a5a392bcca0b None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:58.497058 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10933e4d-2a8b-47ae-ac23-a5a392bcca0b None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:58.497058 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10933e4d-2a8b-47ae-ac23-a5a392bcca0b None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:58.498548 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 510/996] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:58.503448 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6968001-fd8a-47ec-a419-93b5d94f1719 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:58.503858 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6968001-fd8a-47ec-a419-93b5d94f1719 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:58.504668 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6968001-fd8a-47ec-a419-93b5d94f1719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:58.522881 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f6968001-fd8a-47ec-a419-93b5d94f1719 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:58.524272 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f6968001-fd8a-47ec-a419-93b5d94f1719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:58.600519 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 487/997] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:58 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:56:58.620610 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569d7af-c571-4074-96b7-05d535da081d None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:58.621113 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569d7af-c571-4074-96b7-05d535da081d None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:58.621397 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569d7af-c571-4074-96b7-05d535da081d None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:58.621780 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6968001-fd8a-47ec-a419-93b5d94f1719 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:58.622882 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 511/998] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:56:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:56:58.627614 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44160866-5a0f-46e5-8c82-22015d002406 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:58.627904 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44160866-5a0f-46e5-8c82-22015d002406 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:58.628190 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44160866-5a0f-46e5-8c82-22015d002406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:58.649516 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-44160866-5a0f-46e5-8c82-22015d002406 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:56:58.651532 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-44160866-5a0f-46e5-8c82-22015d002406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:56:58.717748 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:58.732920 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 512/999] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:56:58 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:56:58.735029 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6968001-fd8a-47ec-a419-93b5d94f1719 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:58.745562 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:58.755712 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44160866-5a0f-46e5-8c82-22015d002406 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:58.780667 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e576938e-03fd-409a-9ba6-69db3a8b9e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:58.782056 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576938e-03fd-409a-9ba6-69db3a8b9e21 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:58.782593 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576938e-03fd-409a-9ba6-69db3a8b9e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:58.783018 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576938e-03fd-409a-9ba6-69db3a8b9e21 None admin] PATH_INFO: `/v3/services/image` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:58.784367 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e576938e-03fd-409a-9ba6-69db3a8b9e21 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:58.786974 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e576938e-03fd-409a-9ba6-69db3a8b9e21 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:58.800376 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-e576938e-03fd-409a-9ba6-69db3a8b9e21 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 13:56:58.800376 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:58.800376 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:56:58.800376 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:56:58.800376 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:56:58.800376 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:56:58.800376 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:56:58.800376 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:58.800376 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 13:56:58.800376 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:58.800376 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:56:58.800376 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:58.800376 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:58.800376 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:58.800376 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:58.800376 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:58.800376 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:58.801678 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:58.801678 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:58.801678 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:58.801678 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:58.801678 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:58.801678 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:58.801678 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:56:58.801678 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:56:58.801678 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:56:58.801678 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:56:58.801678 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:58.801678 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:58.801678 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:58.801678 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:58.801678 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:58.801678 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:58.802687 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:58.802687 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:58.802687 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:58.802687 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:58.802687 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:58.802687 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:58.802687 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:58.802687 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:58.802687 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:58.802687 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:58.802687 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:56:58.802687 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:58.802687 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 13:56:58.802687 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:56:58.806042 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 488/1000] 10.222.0.48 () {56 vars in 1105 bytes} [Mon Aug 1 13:56:58 2022] GET /identity/v3/services/image => generated 86 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:58.811738 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e576938e-03fd-409a-9ba6-69db3a8b9e21 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:58.867728 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44160866-5a0f-46e5-8c82-22015d002406 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:58.904858 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5f38a-7158-46e8-a215-c1ab6ed1a385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:58.905785 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5f38a-7158-46e8-a215-c1ab6ed1a385 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:58.906221 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5f38a-7158-46e8-a215-c1ab6ed1a385 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:58.906648 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5f38a-7158-46e8-a215-c1ab6ed1a385 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:58.907427 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb5f38a-7158-46e8-a215-c1ab6ed1a385 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:58.908979 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb5f38a-7158-46e8-a215-c1ab6ed1a385 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:58.918194 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-5fb5f38a-7158-46e8-a215-c1ab6ed1a385 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 13:56:58.918194 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:58.918194 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:56:58.918194 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:56:58.918194 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:56:58.918194 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:56:58.918194 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:56:58.918194 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:58.918194 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 13:56:58.918194 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:58.918194 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:56:58.918194 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:58.918194 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:56:58.918194 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:56:58.918194 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:56:58.918194 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:56:58.918194 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:56:58.919328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:56:58.919328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:56:58.919328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:56:58.919328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:56:58.919328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:56:58.919328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:56:58.919328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:56:58.919328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:56:58.919328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:56:58.919328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:56:58.919328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:56:58.919328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:56:58.919328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:56:58.919328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:56:58.919328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:56:58.919328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:56:58.920278 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:56:58.920278 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:56:58.920278 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:56:58.920278 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:56:58.920278 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:56:58.920278 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:56:58.920278 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:56:58.920278 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:56:58.920278 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:56:58.920278 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:56:58.920278 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:56:58.920278 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:56:58.920278 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 13:56:58.920278 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:56:58.922547 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 513/1001] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:56:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:56:58.927866 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e576938e-03fd-409a-9ba6-69db3a8b9e21 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:58.928344 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5f38a-7158-46e8-a215-c1ab6ed1a385 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:58.966927 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181c290-f011-42bc-a45e-6415546cc4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:58.967547 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181c290-f011-42bc-a45e-6415546cc4d3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:58.967829 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181c290-f011-42bc-a45e-6415546cc4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:58.968103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181c290-f011-42bc-a45e-6415546cc4d3 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:58.968819 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a181c290-f011-42bc-a45e-6415546cc4d3 None admin] RBAC: Adding query filter params (name=image) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:58.969125 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a181c290-f011-42bc-a45e-6415546cc4d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:58.970622 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a181c290-f011-42bc-a45e-6415546cc4d3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:58.976151 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 489/1002] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:56:58.981625 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181c290-f011-42bc-a45e-6415546cc4d3 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:58.989003 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:59.018915 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:59.052319 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5f38a-7158-46e8-a215-c1ab6ed1a385 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:59.086261 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19821db4-ed3c-49f0-b421-5e730e84f7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:59.086856 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19821db4-ed3c-49f0-b421-5e730e84f7b1 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:59.087136 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19821db4-ed3c-49f0-b421-5e730e84f7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:59.087589 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19821db4-ed3c-49f0-b421-5e730e84f7b1 None admin] PATH_INFO: `/v3/services` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:59.088077 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19821db4-ed3c-49f0-b421-5e730e84f7b1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:59.088373 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19821db4-ed3c-49f0-b421-5e730e84f7b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:59.089769 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19821db4-ed3c-49f0-b421-5e730e84f7b1 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:59.093901 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181c290-f011-42bc-a45e-6415546cc4d3 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:59.094747 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 514/1003] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 13:56:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:56:59.100856 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19821db4-ed3c-49f0-b421-5e730e84f7b1 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:59.142034 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661cf1a1-22ca-4cdf-a8f2-e9d9bc00b3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:59.142675 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cf1a1-22ca-4cdf-a8f2-e9d9bc00b3e8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:59.142964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cf1a1-22ca-4cdf-a8f2-e9d9bc00b3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:59.143243 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cf1a1-22ca-4cdf-a8f2-e9d9bc00b3e8 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:59.143948 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661cf1a1-22ca-4cdf-a8f2-e9d9bc00b3e8 None admin] RBAC: Adding query filter params (type=image) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:59.144262 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661cf1a1-22ca-4cdf-a8f2-e9d9bc00b3e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:59.146256 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661cf1a1-22ca-4cdf-a8f2-e9d9bc00b3e8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:59.152050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 490/1004] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:56:58 2022] GET /identity/v3/services?type=image => generated 122 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:56:59.221933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19821db4-ed3c-49f0-b421-5e730e84f7b1 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:59.256433 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:59.280607 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917f310-082e-4da7-bc6e-475e504de00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:59.281183 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917f310-082e-4da7-bc6e-475e504de00f None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:59.281469 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917f310-082e-4da7-bc6e-475e504de00f None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:59.281748 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917f310-082e-4da7-bc6e-475e504de00f None admin] PATH_INFO: `/v3/services` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:59.282456 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5917f310-082e-4da7-bc6e-475e504de00f None admin] RBAC: Adding query filter params (type=compute) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:59.282764 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5917f310-082e-4da7-bc6e-475e504de00f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:59.284158 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5917f310-082e-4da7-bc6e-475e504de00f None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:59.290276 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 515/1005] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 13:56:59 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:56:59.296381 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661cf1a1-22ca-4cdf-a8f2-e9d9bc00b3e8 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:59.408223 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:56:59.412514 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661cf1a1-22ca-4cdf-a8f2-e9d9bc00b3e8 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:59.448251 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b220b135-59a5-4f80-adc3-e008e7989423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:59.448908 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220b135-59a5-4f80-adc3-e008e7989423 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:59.449197 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220b135-59a5-4f80-adc3-e008e7989423 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:59.449480 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220b135-59a5-4f80-adc3-e008e7989423 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:59.450240 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b220b135-59a5-4f80-adc3-e008e7989423 None admin] RBAC: Adding query filter params (interface=public, service_id=48acd9c7cad44a7aa2631799228d5c2f, region_id=RegionOne) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:56:59.450555 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b220b135-59a5-4f80-adc3-e008e7989423 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:59.451918 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b220b135-59a5-4f80-adc3-e008e7989423 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:59.457263 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 491/1006] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Aug 1 13:56:59 2022] GET /identity/v3/endpoints?service_id=48acd9c7cad44a7aa2631799228d5c2f&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:56:59.462854 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917f310-082e-4da7-bc6e-475e504de00f None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:56:59.581331 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917f310-082e-4da7-bc6e-475e504de00f None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:56:59.615249 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2276b23-d93c-40ab-adbe-baacd0a87338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:56:59.615870 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2276b23-d93c-40ab-adbe-baacd0a87338 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:56:59.616150 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2276b23-d93c-40ab-adbe-baacd0a87338 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:56:59.616426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2276b23-d93c-40ab-adbe-baacd0a87338 None admin] PATH_INFO: `/v3/services` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:56:59.617110 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2276b23-d93c-40ab-adbe-baacd0a87338 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:56:59.618470 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2276b23-d93c-40ab-adbe-baacd0a87338 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:56:59.628538 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 516/1007] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Aug 1 13:56:59 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 13:57:00.272828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df85be9-6cc9-4feb-95a3-3bd7ba531112 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:00.273793 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df85be9-6cc9-4feb-95a3-3bd7ba531112 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:00.274272 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df85be9-6cc9-4feb-95a3-3bd7ba531112 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:00.276230 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 492/1008] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:57:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:57:00.281856 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942db94-81a4-4a5f-84bf-0b4b5b27be55 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:00.282173 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942db94-81a4-4a5f-84bf-0b4b5b27be55 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:00.282433 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942db94-81a4-4a5f-84bf-0b4b5b27be55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:00.302152 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-e942db94-81a4-4a5f-84bf-0b4b5b27be55 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:57:00.303356 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-e942db94-81a4-4a5f-84bf-0b4b5b27be55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:57:00.345424 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:00.374599 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 517/1009] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:57:00 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:57:00.398427 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df85be9-6cc9-4feb-95a3-3bd7ba531112 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:00.500530 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df85be9-6cc9-4feb-95a3-3bd7ba531112 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:00.534582 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d678f4a-36c3-4cd3-b60a-ad9f6cb72320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:00.535493 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d678f4a-36c3-4cd3-b60a-ad9f6cb72320 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:00.535986 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d678f4a-36c3-4cd3-b60a-ad9f6cb72320 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:00.536537 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d678f4a-36c3-4cd3-b60a-ad9f6cb72320 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:00.537924 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d678f4a-36c3-4cd3-b60a-ad9f6cb72320 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:00.539725 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d678f4a-36c3-4cd3-b60a-ad9f6cb72320 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:00.541915 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 493/1010] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:57:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:00.547462 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942db94-81a4-4a5f-84bf-0b4b5b27be55 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:00.648635 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942db94-81a4-4a5f-84bf-0b4b5b27be55 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:00.682420 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf74fedb-62f2-4849-88f1-557bcd2c643c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:00.683054 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74fedb-62f2-4849-88f1-557bcd2c643c None admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:00.683339 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74fedb-62f2-4849-88f1-557bcd2c643c None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:00.683618 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74fedb-62f2-4849-88f1-557bcd2c643c None admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:00.684292 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf74fedb-62f2-4849-88f1-557bcd2c643c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:00.685762 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf74fedb-62f2-4849-88f1-557bcd2c643c None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:00.715477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 518/1011] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 13:57:00 2022] POST /identity/v3/projects => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:57:01.177116 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8671134-f702-4810-863b-f54a1897bc59 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:01.177492 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8671134-f702-4810-863b-f54a1897bc59 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:01.177687 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8671134-f702-4810-863b-f54a1897bc59 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:01.179075 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 494/1012] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:57:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:57:01.184090 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea6bb3-37f7-4c2c-bec1-30d6c0e7a7b1 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:01.184387 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea6bb3-37f7-4c2c-bec1-30d6c0e7a7b1 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:01.184706 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea6bb3-37f7-4c2c-bec1-30d6c0e7a7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:01.202948 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-b3ea6bb3-37f7-4c2c-bec1-30d6c0e7a7b1 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:57:01.204138 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-b3ea6bb3-37f7-4c2c-bec1-30d6c0e7a7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:57:01.226067 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:01.277159 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 519/1013] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:57:01 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:57:01.299223 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8671134-f702-4810-863b-f54a1897bc59 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:01.333231 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:01.407095 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8671134-f702-4810-863b-f54a1897bc59 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:01.441627 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c53fdb-3b1e-4f50-8598-1ea691bbb154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:01.442198 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c53fdb-3b1e-4f50-8598-1ea691bbb154 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:01.442479 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c53fdb-3b1e-4f50-8598-1ea691bbb154 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:01.442775 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c53fdb-3b1e-4f50-8598-1ea691bbb154 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:01.443511 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c53fdb-3b1e-4f50-8598-1ea691bbb154 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:01.444901 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c53fdb-3b1e-4f50-8598-1ea691bbb154 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:01.462349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 495/1014] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 13:57:01 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:57:01.643371 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920be8e7-1959-429d-984a-40fe73e14277 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:01.643648 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920be8e7-1959-429d-984a-40fe73e14277 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:01.643927 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920be8e7-1959-429d-984a-40fe73e14277 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:01.646020 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 520/1015] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:57:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:57:01.650934 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62ab02-54e6-4ee9-9d26-7f2a9d0fcd7c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:01.651231 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62ab02-54e6-4ee9-9d26-7f2a9d0fcd7c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:01.651516 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62ab02-54e6-4ee9-9d26-7f2a9d0fcd7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:01.670037 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8f62ab02-54e6-4ee9-9d26-7f2a9d0fcd7c None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:57:01.671225 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8f62ab02-54e6-4ee9-9d26-7f2a9d0fcd7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:57:01.742521 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 496/1016] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:57:01 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:57:01.764674 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920be8e7-1959-429d-984a-40fe73e14277 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:01.878210 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920be8e7-1959-429d-984a-40fe73e14277 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:01.913266 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05607373-7d5c-4229-88ab-8c6525ee44d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:01.913914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05607373-7d5c-4229-88ab-8c6525ee44d0 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:01.914197 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05607373-7d5c-4229-88ab-8c6525ee44d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:01.914473 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05607373-7d5c-4229-88ab-8c6525ee44d0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:01.915100 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05607373-7d5c-4229-88ab-8c6525ee44d0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:01.916466 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05607373-7d5c-4229-88ab-8c6525ee44d0 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:01.925317 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-05607373-7d5c-4229-88ab-8c6525ee44d0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 13:57:01.925317 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:01.925317 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:57:01.925317 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:57:01.925317 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:57:01.925317 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:57:01.925317 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:57:01.925317 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:57:01.925317 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 13:57:01.925317 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:57:01.925317 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:57:01.925317 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:57:01.925317 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:01.925317 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:01.925317 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:01.925317 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:01.925317 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:01.926291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:01.926291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:01.926291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:01.926291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:01.926291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:01.926291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:01.926291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:57:01.926291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:57:01.926291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:57:01.926291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:57:01.926291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:01.926291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:01.926291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:01.926291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:01.926291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:01.926291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:01.927114 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:01.927114 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:01.927114 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:01.927114 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:01.927114 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:01.927114 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:01.927114 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:01.927114 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:01.927114 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:01.927114 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:01.927114 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:57:01.927114 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:57:01.927114 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 13:57:01.927114 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:57:01.928798 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 521/1017] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:57:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:01.932420 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f62ab02-54e6-4ee9-9d26-7f2a9d0fcd7c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:01.952580 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:02.035431 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f62ab02-54e6-4ee9-9d26-7f2a9d0fcd7c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:02.069461 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9204893d-9745-4c6f-aad4-ef80db9144ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:02.070166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204893d-9745-4c6f-aad4-ef80db9144ba None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:02.070496 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204893d-9745-4c6f-aad4-ef80db9144ba None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:02.070778 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204893d-9745-4c6f-aad4-ef80db9144ba None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:02.071456 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9204893d-9745-4c6f-aad4-ef80db9144ba None admin] RBAC: Adding query filter params (name=compute) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:02.071761 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9204893d-9745-4c6f-aad4-ef80db9144ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:02.073335 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9204893d-9745-4c6f-aad4-ef80db9144ba None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:02.083596 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 497/1018] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 13:57:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:02.084803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05607373-7d5c-4229-88ab-8c6525ee44d0 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:02.107036 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:02.186849 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05607373-7d5c-4229-88ab-8c6525ee44d0 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:02.220611 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd1f7a5-af5b-48ce-8882-fa1657550548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:02.221242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1f7a5-af5b-48ce-8882-fa1657550548 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:02.221525 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1f7a5-af5b-48ce-8882-fa1657550548 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:02.221802 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1f7a5-af5b-48ce-8882-fa1657550548 None admin] PATH_INFO: `/v3/services` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:02.222515 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd1f7a5-af5b-48ce-8882-fa1657550548 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:02.222825 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd1f7a5-af5b-48ce-8882-fa1657550548 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:02.224184 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd1f7a5-af5b-48ce-8882-fa1657550548 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:02.235282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 522/1019] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 13:57:02 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:02.235439 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9204893d-9745-4c6f-aad4-ef80db9144ba None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:02.322485 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:02.343072 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9204893d-9745-4c6f-aad4-ef80db9144ba None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:02.377368 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b8e0d3-3cc5-425d-ac51-a67ea32903f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:02.377991 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8e0d3-3cc5-425d-ac51-a67ea32903f5 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:02.378281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8e0d3-3cc5-425d-ac51-a67ea32903f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:02.378562 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8e0d3-3cc5-425d-ac51-a67ea32903f5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:02.379839 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b8e0d3-3cc5-425d-ac51-a67ea32903f5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:02.381229 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b8e0d3-3cc5-425d-ac51-a67ea32903f5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:02.408518 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 498/1020] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 13:57:02 2022] POST /identity/v3/endpoints => generated 334 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 13:57:02.750820 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057873d0-271f-4f3e-b4d4-4cb52ad35aae None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:02.751147 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057873d0-271f-4f3e-b4d4-4cb52ad35aae None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:02.751379 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057873d0-271f-4f3e-b4d4-4cb52ad35aae None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:02.752906 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 523/1021] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:57:02.758071 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70817b-af7c-4d86-95fa-950dec63d2ce None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:02.758365 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70817b-af7c-4d86-95fa-950dec63d2ce None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:02.758645 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70817b-af7c-4d86-95fa-950dec63d2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:02.776204 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-fe70817b-af7c-4d86-95fa-950dec63d2ce None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:57:02.777404 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fe70817b-af7c-4d86-95fa-950dec63d2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:57:02.856257 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 499/1022] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:57:02 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:57:02.881653 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057873d0-271f-4f3e-b4d4-4cb52ad35aae None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:02.970292 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:02.988522 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057873d0-271f-4f3e-b4d4-4cb52ad35aae None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:03.022623 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e7e65-2c90-4b54-805f-2b362044769b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:03.023193 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e7e65-2c90-4b54-805f-2b362044769b None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:03.023480 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e7e65-2c90-4b54-805f-2b362044769b None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:03.023762 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e7e65-2c90-4b54-805f-2b362044769b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:03.024961 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e7e65-2c90-4b54-805f-2b362044769b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:03.026431 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e7e65-2c90-4b54-805f-2b362044769b None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:03.032704 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 524/1023] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:57:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:03.033881 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe70817b-af7c-4d86-95fa-950dec63d2ce None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:03.101755 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:03.143166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe70817b-af7c-4d86-95fa-950dec63d2ce None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:03.177883 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb386d7-644d-4412-82d3-ab389180784b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:03.178509 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb386d7-644d-4412-82d3-ab389180784b None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:03.178792 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb386d7-644d-4412-82d3-ab389180784b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:03.179067 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb386d7-644d-4412-82d3-ab389180784b None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:03.179765 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb386d7-644d-4412-82d3-ab389180784b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:03.181282 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb386d7-644d-4412-82d3-ab389180784b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:03.204030 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 500/1024] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 13:57:03 2022] POST /identity/v3/users => generated 285 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:57:03.510260 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ade70d-a270-4f14-ab4b-568e86c09751 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:03.510622 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ade70d-a270-4f14-ab4b-568e86c09751 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:03.510829 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ade70d-a270-4f14-ab4b-568e86c09751 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:03.512331 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 525/1025] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:57:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:57:03.517418 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed1a3c-8f78-4e60-8be4-0e8a006cd584 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:03.517847 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed1a3c-8f78-4e60-8be4-0e8a006cd584 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:03.518015 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed1a3c-8f78-4e60-8be4-0e8a006cd584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:03.536716 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-57ed1a3c-8f78-4e60-8be4-0e8a006cd584 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:57:03.537909 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-57ed1a3c-8f78-4e60-8be4-0e8a006cd584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:57:03.607446 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 501/1026] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:57:03 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:57:03.630384 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ade70d-a270-4f14-ab4b-568e86c09751 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:03.733830 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ade70d-a270-4f14-ab4b-568e86c09751 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:03.768193 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49242275-2b15-4c17-88af-f0bed9265c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:03.768814 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49242275-2b15-4c17-88af-f0bed9265c19 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:03.769101 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49242275-2b15-4c17-88af-f0bed9265c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:03.769406 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49242275-2b15-4c17-88af-f0bed9265c19 None admin] PATH_INFO: `/v3/services/image` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:03.770038 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49242275-2b15-4c17-88af-f0bed9265c19 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:03.771401 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49242275-2b15-4c17-88af-f0bed9265c19 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:03.780187 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-49242275-2b15-4c17-88af-f0bed9265c19 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 13:57:03.780187 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:03.780187 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:57:03.780187 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:57:03.780187 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:57:03.780187 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:57:03.780187 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:57:03.780187 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:57:03.780187 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 13:57:03.780187 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:57:03.780187 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:57:03.780187 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:57:03.780187 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:03.780187 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:03.780187 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:03.780187 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:03.780187 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:03.781356 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:03.781356 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:03.781356 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:03.781356 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:03.781356 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:03.781356 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:03.781356 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:57:03.781356 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:57:03.781356 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:57:03.781356 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:57:03.781356 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:03.781356 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:03.781356 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:03.781356 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:03.781356 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:03.781356 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:03.782224 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:03.782224 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:03.782224 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:03.782224 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:03.782224 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:03.782224 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:03.782224 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:03.782224 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:03.782224 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:03.782224 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:03.782224 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:57:03.782224 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:57:03.782224 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 13:57:03.782224 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:57:03.782224 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 526/1027] 10.222.0.48 () {56 vars in 1105 bytes} [Mon Aug 1 13:57:03 2022] GET /identity/v3/services/image => generated 86 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:03.788515 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed1a3c-8f78-4e60-8be4-0e8a006cd584 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:03.844126 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:03.896135 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed1a3c-8f78-4e60-8be4-0e8a006cd584 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:03.930210 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5a4f5-ce81-4d1f-a7c5-b111804c732a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:03.930842 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5a4f5-ce81-4d1f-a7c5-b111804c732a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:03.931126 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5a4f5-ce81-4d1f-a7c5-b111804c732a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:03.931402 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5a4f5-ce81-4d1f-a7c5-b111804c732a None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:03.932089 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf5a4f5-ce81-4d1f-a7c5-b111804c732a None admin] RBAC: Adding query filter params (name=image) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:03.932400 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf5a4f5-ce81-4d1f-a7c5-b111804c732a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:03.933788 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf5a4f5-ce81-4d1f-a7c5-b111804c732a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:03.944271 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 502/1028] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:57:03 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:03.945561 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49242275-2b15-4c17-88af-f0bed9265c19 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:03.964409 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:04.048536 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49242275-2b15-4c17-88af-f0bed9265c19 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:04.082905 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92270306-d9fe-4729-a91d-da151c268ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:04.083526 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92270306-d9fe-4729-a91d-da151c268ba8 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:04.083814 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92270306-d9fe-4729-a91d-da151c268ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:04.084089 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92270306-d9fe-4729-a91d-da151c268ba8 None admin] PATH_INFO: `/v3/services` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:04.084798 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92270306-d9fe-4729-a91d-da151c268ba8 None admin] RBAC: Adding query filter params (type=image) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:04.085107 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92270306-d9fe-4729-a91d-da151c268ba8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:04.086470 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92270306-d9fe-4729-a91d-da151c268ba8 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:04.096080 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 527/1029] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:57:03 2022] GET /identity/v3/services?type=image => generated 350 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:04.097293 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5a4f5-ce81-4d1f-a7c5-b111804c732a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:04.203579 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5a4f5-ce81-4d1f-a7c5-b111804c732a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:04.218550 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:04.239680 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0e165-e499-4fe5-af7b-d38318cbb134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:04.240305 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0e165-e499-4fe5-af7b-d38318cbb134 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:04.240620 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0e165-e499-4fe5-af7b-d38318cbb134 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:04.240899 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0e165-e499-4fe5-af7b-d38318cbb134 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:04.241661 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e0e165-e499-4fe5-af7b-d38318cbb134 None admin] RBAC: Adding query filter params (interface=public, service_id=437d09b128624bceae4234e537a4c294, region_id=RegionOne) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:04.241974 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e0e165-e499-4fe5-af7b-d38318cbb134 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:04.243329 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e0e165-e499-4fe5-af7b-d38318cbb134 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:04.252977 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 503/1030] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Aug 1 13:57:04 2022] GET /identity/v3/endpoints?service_id=437d09b128624bceae4234e537a4c294&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:57:04.254979 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92270306-d9fe-4729-a91d-da151c268ba8 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:04.343717 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:04.366528 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92270306-d9fe-4729-a91d-da151c268ba8 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:04.403260 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05642a35-893f-4b9b-bf26-9427d2306d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:04.403896 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05642a35-893f-4b9b-bf26-9427d2306d96 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:04.404181 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05642a35-893f-4b9b-bf26-9427d2306d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:04.404460 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05642a35-893f-4b9b-bf26-9427d2306d96 None admin] PATH_INFO: `/v3/services` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:04.405148 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05642a35-893f-4b9b-bf26-9427d2306d96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:04.406518 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05642a35-893f-4b9b-bf26-9427d2306d96 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:04.420755 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 528/1031] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Aug 1 13:57:04 2022] GET /identity/v3/services => generated 1957 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 13:57:04.513724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66406f24-6b62-4cc1-9e98-6dc9c19684b3 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:04.514076 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66406f24-6b62-4cc1-9e98-6dc9c19684b3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:04.514302 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66406f24-6b62-4cc1-9e98-6dc9c19684b3 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:04.515850 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 504/1032] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:57:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:57:04.521253 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9134be-6511-49ab-9d0d-d6bbbe1aafb0 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:04.521543 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9134be-6511-49ab-9d0d-d6bbbe1aafb0 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:04.521815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9134be-6511-49ab-9d0d-d6bbbe1aafb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:04.540166 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-eb9134be-6511-49ab-9d0d-d6bbbe1aafb0 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:57:04.541414 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-eb9134be-6511-49ab-9d0d-d6bbbe1aafb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:57:04.612723 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 529/1033] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:57:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:57:04.635992 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66406f24-6b62-4cc1-9e98-6dc9c19684b3 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:04.737482 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66406f24-6b62-4cc1-9e98-6dc9c19684b3 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:04.772147 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c46818-db11-44f4-b92e-9f9530d2ba9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:04.772820 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c46818-db11-44f4-b92e-9f9530d2ba9f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:04.773106 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c46818-db11-44f4-b92e-9f9530d2ba9f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:04.773382 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c46818-db11-44f4-b92e-9f9530d2ba9f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:04.783667 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c46818-db11-44f4-b92e-9f9530d2ba9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:57:04.789108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-c3c46818-db11-44f4-b92e-9f9530d2ba9f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 13:57:04.789108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:04.789108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:04.789108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:04.789108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:04.789108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:04.789108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:04.789108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:04.789108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:04.789108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:04.789108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:04.789108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:04.789108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:57:04.789108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:57:04.789108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:57:04.789108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:57:04.790012 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:57:04.790012 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:57:04.790012 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:57:04.790012 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:57:04.790012 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:04.790012 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:04.790012 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:04.790012 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:04.790012 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:04.790012 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:04.790012 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:04.790012 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:04.790012 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:04.790012 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:04.790012 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:04.790012 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:04.790854 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:04.790854 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:04.790854 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:04.790854 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:04.790854 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:57:04.790854 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:57:04.790854 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:57:04.790854 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:57:04.790854 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:57:04.790854 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:57:04.790854 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 13:57:04.790854 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:57:04.792568 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 505/1034] 10.222.0.48 () {56 vars in 1120 bytes} [Mon Aug 1 13:57:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:04.796227 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9134be-6511-49ab-9d0d-d6bbbe1aafb0 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:04.881622 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:04.907300 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9134be-6511-49ab-9d0d-d6bbbe1aafb0 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:04.941856 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be5911-69c9-4ef8-ac0a-54bf9c7b21d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:04.942480 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be5911-69c9-4ef8-ac0a-54bf9c7b21d9 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:04.942758 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be5911-69c9-4ef8-ac0a-54bf9c7b21d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:04.943034 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be5911-69c9-4ef8-ac0a-54bf9c7b21d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:04.943751 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28be5911-69c9-4ef8-ac0a-54bf9c7b21d9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:04.944052 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28be5911-69c9-4ef8-ac0a-54bf9c7b21d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:04.945431 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28be5911-69c9-4ef8-ac0a-54bf9c7b21d9 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:04.958230 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 530/1035] 10.222.0.48 () {56 vars in 1115 bytes} [Mon Aug 1 13:57:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:04.959192 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c46818-db11-44f4-b92e-9f9530d2ba9f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:04.999955 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:05.061012 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c46818-db11-44f4-b92e-9f9530d2ba9f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:05.099557 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a84c5d-e991-41aa-a527-201c49a675bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:05.100212 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a84c5d-e991-41aa-a527-201c49a675bc None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:05.100515 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a84c5d-e991-41aa-a527-201c49a675bc None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:05.100790 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a84c5d-e991-41aa-a527-201c49a675bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:05.111225 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a84c5d-e991-41aa-a527-201c49a675bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:05.112682 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a84c5d-e991-41aa-a527-201c49a675bc None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:05.127631 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-f3a84c5d-e991-41aa-a527-201c49a675bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:57:05.127631 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:05.127631 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:05.127631 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:05.127631 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:05.127631 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:05.127631 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:05.127631 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:05.127631 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:05.127631 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:05.127631 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:05.127631 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:05.127631 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:57:05.127631 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:57:05.127631 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:57:05.129470 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:57:05.129470 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:05.129470 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:05.129470 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:05.129470 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:05.129470 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:05.129470 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:05.129470 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:05.129470 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:05.129470 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:05.129470 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:05.129470 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:05.129470 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:05.129470 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:05.129470 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:05.129470 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:05.129470 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:05.130369 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:57:05.130369 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:57:05.130369 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:57:05.130369 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:57:05.130369 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 506/1036] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:57:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:57:05.134931 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be5911-69c9-4ef8-ac0a-54bf9c7b21d9 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:05.234523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be5911-69c9-4ef8-ac0a-54bf9c7b21d9 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:05.267450 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ece31-6e46-429b-a2ae-aa2d430eb323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:05.268066 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ece31-6e46-429b-a2ae-aa2d430eb323 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:05.268346 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ece31-6e46-429b-a2ae-aa2d430eb323 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:05.268716 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ece31-6e46-429b-a2ae-aa2d430eb323 None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:05.269974 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8ece31-6e46-429b-a2ae-aa2d430eb323 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:05.270215 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8ece31-6e46-429b-a2ae-aa2d430eb323 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:05.271572 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8ece31-6e46-429b-a2ae-aa2d430eb323 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:05.273932 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:05.281432 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec8396-193d-4e76-93ae-3fb1af31cf5a None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:05.281793 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec8396-193d-4e76-93ae-3fb1af31cf5a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:05.282149 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec8396-193d-4e76-93ae-3fb1af31cf5a None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:05.283618 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 507/1037] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:57:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:57:05.288761 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 531/1038] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:57:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:05.289740 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ad6b2-ce07-4e2f-99d4-d32f4da73cb7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:05.290100 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ad6b2-ce07-4e2f-99d4-d32f4da73cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:05.290484 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ad6b2-ce07-4e2f-99d4-d32f4da73cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:05.291686 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ece31-6e46-429b-a2ae-aa2d430eb323 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:05.308976 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5c4ad6b2-ce07-4e2f-99d4-d32f4da73cb7 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:57:05.310169 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5c4ad6b2-ce07-4e2f-99d4-d32f4da73cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:57:05.382114 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 508/1039] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:57:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:57:05.403159 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ece31-6e46-429b-a2ae-aa2d430eb323 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:05.405200 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ad6b2-ce07-4e2f-99d4-d32f4da73cb7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:05.427504 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:05.440368 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4ad2f-aa0c-4df9-bc9d-d0c558931d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:05.441065 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4ad2f-aa0c-4df9-bc9d-d0c558931d87 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:05.441352 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4ad2f-aa0c-4df9-bc9d-d0c558931d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:05.441628 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4ad2f-aa0c-4df9-bc9d-d0c558931d87 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:05.448927 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b4ad2f-aa0c-4df9-bc9d-d0c558931d87 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:05.450360 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b4ad2f-aa0c-4df9-bc9d-d0c558931d87 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:05.464003 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-94b4ad2f-aa0c-4df9-bc9d-d0c558931d87 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 13:57:05.464003 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:05.464003 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:05.464003 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:05.464003 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:05.464003 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:05.464003 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:05.464003 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:05.464003 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:05.464003 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:05.464003 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:05.464003 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:05.464003 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:57:05.464003 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:57:05.464003 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:57:05.464003 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:57:05.465119 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:05.465119 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:05.465119 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:57:05.465119 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:57:05.465119 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:57:05.465119 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:57:05.465119 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:05.465119 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:05.465119 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:05.465119 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:05.465119 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:05.465119 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:05.465119 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:05.465119 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:05.465119 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:05.465119 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:05.465119 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:05.465119 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:05.466053 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:05.466053 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:05.466053 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:57:05.466053 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:57:05.466053 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:57:05.466053 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:57:05.466053 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:57:05.466053 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:57:05.466053 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 13:57:05.466053 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:57:05.467392 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 532/1040] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Aug 1 13:57:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:57:05.470868 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4ad2f-aa0c-4df9-bc9d-d0c558931d87 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:05.513300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ad6b2-ce07-4e2f-99d4-d32f4da73cb7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:05.530827 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:05.550992 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e08b16-ca7d-4b69-82ac-ae8249706f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:05.551657 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e08b16-ca7d-4b69-82ac-ae8249706f27 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:05.551942 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e08b16-ca7d-4b69-82ac-ae8249706f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:05.552222 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e08b16-ca7d-4b69-82ac-ae8249706f27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:05.564126 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e08b16-ca7d-4b69-82ac-ae8249706f27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:57:05.570103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-42e08b16-ca7d-4b69-82ac-ae8249706f27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:57:05.570103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:05.570103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:05.570103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:05.570103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:05.570103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:05.570103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:05.570103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:05.570103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:05.570103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:05.570103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:05.570103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:05.570103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:57:05.570103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:57:05.570103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:57:05.570103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:57:05.571017 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:57:05.571017 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:57:05.571017 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:57:05.571017 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:57:05.571017 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:05.571017 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:05.571017 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:05.571017 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:05.571017 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:05.571017 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:05.571017 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:05.571017 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:05.571017 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:05.571017 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:05.571017 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:05.571017 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:05.571837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:05.571837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:05.571837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:05.571837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:05.571837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:57:05.571837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:57:05.571837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:57:05.571837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:57:05.571837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:57:05.571837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:57:05.571837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:57:05.571837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:57:05.571837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 509/1041] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 13:57:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:57:05.577117 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e08b16-ca7d-4b69-82ac-ae8249706f27 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:05.581240 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4ad2f-aa0c-4df9-bc9d-d0c558931d87 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:05.617123 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc70edb2-34ee-4d5f-8754-9a47b88cc6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:05.617750 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70edb2-34ee-4d5f-8754-9a47b88cc6da None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:05.618036 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70edb2-34ee-4d5f-8754-9a47b88cc6da None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:05.618316 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70edb2-34ee-4d5f-8754-9a47b88cc6da None admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:05.619128 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc70edb2-34ee-4d5f-8754-9a47b88cc6da None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:05.619444 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc70edb2-34ee-4d5f-8754-9a47b88cc6da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:05.620897 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc70edb2-34ee-4d5f-8754-9a47b88cc6da None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:05.640697 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 533/1042] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Aug 1 13:57:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:57:05.647035 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc70edb2-34ee-4d5f-8754-9a47b88cc6da None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:05.669917 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:05.690083 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e08b16-ca7d-4b69-82ac-ae8249706f27 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:05.723636 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d7c39-35af-4664-b28a-b4550fed4687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:05.724249 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d7c39-35af-4664-b28a-b4550fed4687 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:05.724554 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d7c39-35af-4664-b28a-b4550fed4687 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:05.724831 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d7c39-35af-4664-b28a-b4550fed4687 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:05.725540 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116d7c39-35af-4664-b28a-b4550fed4687 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:05.725843 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116d7c39-35af-4664-b28a-b4550fed4687 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:05.727217 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116d7c39-35af-4664-b28a-b4550fed4687 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:05.735739 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 510/1043] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:57:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:57:05.742182 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d7c39-35af-4664-b28a-b4550fed4687 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:05.760044 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc70edb2-34ee-4d5f-8754-9a47b88cc6da None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:05.774746 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:05.795111 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7898474-21dd-4c77-9130-0848935fb84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:05.795732 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7898474-21dd-4c77-9130-0848935fb84f None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:05.796012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7898474-21dd-4c77-9130-0848935fb84f None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:05.796286 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7898474-21dd-4c77-9130-0848935fb84f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:05.853837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d7c39-35af-4664-b28a-b4550fed4687 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:05.868703 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7898474-21dd-4c77-9130-0848935fb84f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:05.870308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7898474-21dd-4c77-9130-0848935fb84f None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:05.887950 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dabd9c-0e3d-4788-9277-7689abca88a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:05.888629 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dabd9c-0e3d-4788-9277-7689abca88a6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:05.888909 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dabd9c-0e3d-4788-9277-7689abca88a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:05.889183 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dabd9c-0e3d-4788-9277-7689abca88a6 None admin] PATH_INFO: `/v3/users/747829c276f9498989a49a1324c936a7` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:05.900588 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:05.903710 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dabd9c-0e3d-4788-9277-7689abca88a6 None admin] RBAC: Authorizing `identity:get_user(user_id=747829c276f9498989a49a1324c936a7)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:05.905263 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dabd9c-0e3d-4788-9277-7689abca88a6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:05.911832 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 511/1044] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:57:05 2022] GET /identity/v3/users/747829c276f9498989a49a1324c936a7 => generated 285 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:57:05.913001 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dabd9c-0e3d-4788-9277-7689abca88a6 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:06.001153 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 534/1045] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:57:05 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 13:57:06.007421 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7898474-21dd-4c77-9130-0848935fb84f None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:06.022313 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:06.028546 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dabd9c-0e3d-4788-9277-7689abca88a6 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:06.066824 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eae0fe-5746-446a-993e-793d4447658c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:06.067727 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eae0fe-5746-446a-993e-793d4447658c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:06.068193 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eae0fe-5746-446a-993e-793d4447658c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:06.068680 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eae0fe-5746-446a-993e-793d4447658c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:06.121181 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7898474-21dd-4c77-9130-0848935fb84f None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:06.134398 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:06.136994 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eae0fe-5746-446a-993e-793d4447658c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:06.138499 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eae0fe-5746-446a-993e-793d4447658c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:06.159410 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0310df50-3999-4472-ac61-9be2cbb2f501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:06.160632 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310df50-3999-4472-ac61-9be2cbb2f501 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:06.161299 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310df50-3999-4472-ac61-9be2cbb2f501 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:06.161608 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310df50-3999-4472-ac61-9be2cbb2f501 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:06.172531 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0310df50-3999-4472-ac61-9be2cbb2f501 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:06.173975 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0310df50-3999-4472-ac61-9be2cbb2f501 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:06.192427 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-0310df50-3999-4472-ac61-9be2cbb2f501 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:57:06.192427 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:06.192427 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:06.192427 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:06.192427 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:06.192427 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:06.192427 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:06.192427 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:06.192427 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:06.192427 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:06.192427 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:06.192427 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:06.192427 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:57:06.192427 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:57:06.192427 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:57:06.193357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:57:06.193357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:06.193357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:06.193357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:06.193357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:06.193357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:06.193357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:06.193357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:06.193357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:06.193357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:06.193357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:06.193357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:06.193357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:06.193357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:06.193357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:06.193357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:06.193357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:06.194270 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:57:06.194270 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:57:06.194270 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:57:06.194270 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:57:06.195095 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 535/1046] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:57:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:57:06.199524 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0310df50-3999-4472-ac61-9be2cbb2f501 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:06.253083 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 512/1047] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:57:05 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 13:57:06.259340 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eae0fe-5746-446a-993e-793d4447658c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:06.267417 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:06.313174 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0310df50-3999-4472-ac61-9be2cbb2f501 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:06.351243 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba37cf5-9bc4-4830-9b64-b72a6789ae73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:06.351873 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba37cf5-9bc4-4830-9b64-b72a6789ae73 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:06.352159 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba37cf5-9bc4-4830-9b64-b72a6789ae73 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:06.352447 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba37cf5-9bc4-4830-9b64-b72a6789ae73 None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:06.353183 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba37cf5-9bc4-4830-9b64-b72a6789ae73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:06.353493 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba37cf5-9bc4-4830-9b64-b72a6789ae73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:06.354876 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba37cf5-9bc4-4830-9b64-b72a6789ae73 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:06.367892 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 536/1048] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:57:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:57:06.373178 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba37cf5-9bc4-4830-9b64-b72a6789ae73 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:06.382746 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:06.386104 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eae0fe-5746-446a-993e-793d4447658c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:06.424147 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b011ea03-8426-4161-899f-09a30fba2e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:06.424826 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011ea03-8426-4161-899f-09a30fba2e8d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:06.425433 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011ea03-8426-4161-899f-09a30fba2e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:06.425557 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011ea03-8426-4161-899f-09a30fba2e8d None admin] PATH_INFO: `/v3/projects/0f7ee50e24ca44fb9ebded4e154bb2f1` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:06.426861 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b011ea03-8426-4161-899f-09a30fba2e8d None admin] RBAC: Authorizing `identity:get_project(project_id=0f7ee50e24ca44fb9ebded4e154bb2f1)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:06.428375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b011ea03-8426-4161-899f-09a30fba2e8d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:06.435446 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 513/1049] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:57:06 2022] GET /identity/v3/projects/0f7ee50e24ca44fb9ebded4e154bb2f1 => generated 311 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:57:06.436080 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b011ea03-8426-4161-899f-09a30fba2e8d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:06.476709 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba37cf5-9bc4-4830-9b64-b72a6789ae73 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:06.501986 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:06.513231 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50b82e6-ef07-41b1-bb9c-3d581e3f58c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:06.513809 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b82e6-ef07-41b1-bb9c-3d581e3f58c8 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:06.514401 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b82e6-ef07-41b1-bb9c-3d581e3f58c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:06.514679 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b82e6-ef07-41b1-bb9c-3d581e3f58c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:06.522343 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50b82e6-ef07-41b1-bb9c-3d581e3f58c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:06.523769 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50b82e6-ef07-41b1-bb9c-3d581e3f58c8 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:06.534889 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-e50b82e6-ef07-41b1-bb9c-3d581e3f58c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:57:06.534889 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:06.534889 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:06.534889 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:06.534889 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:06.534889 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:06.534889 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:06.534889 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:06.534889 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:06.534889 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:06.534889 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:06.534889 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:06.534889 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:57:06.534889 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:57:06.534889 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:57:06.535818 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:57:06.535818 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:06.535818 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:06.535818 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:06.535818 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:06.535818 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:06.535818 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:06.535818 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:06.535818 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:06.535818 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:06.535818 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:06.535818 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:06.535818 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:06.535818 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:06.535818 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:06.535818 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:06.535818 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:06.537150 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:57:06.537150 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:57:06.537150 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:57:06.537150 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:57:06.537150 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:57:06.537150 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:57:06.537150 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:57:06.537150 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:57:06.537584 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b011ea03-8426-4161-899f-09a30fba2e8d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:06.537658 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 537/1050] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:57:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:57:06.537724 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefef75-349d-4cb5-b5fd-985fa3cbaa89 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:06.537986 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefef75-349d-4cb5-b5fd-985fa3cbaa89 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:06.538269 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefef75-349d-4cb5-b5fd-985fa3cbaa89 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:06.539619 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 538/1051] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:57:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:57:06.541870 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfefef75-349d-4cb5-b5fd-985fa3cbaa89 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:06.570497 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce9643-947b-4c9e-a01d-5f00342ebcd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:06.571114 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce9643-947b-4c9e-a01d-5f00342ebcd9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:06.571402 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce9643-947b-4c9e-a01d-5f00342ebcd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:06.571678 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce9643-947b-4c9e-a01d-5f00342ebcd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:06.572468 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ce9643-947b-4c9e-a01d-5f00342ebcd9 None admin] RBAC: Adding query filter params (role.id=58f91e3a13664dabb18c6227a1658fa6, scope.project.id=0f7ee50e24ca44fb9ebded4e154bb2f1, user.id=747829c276f9498989a49a1324c936a7) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:06.572806 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ce9643-947b-4c9e-a01d-5f00342ebcd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:06.574221 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ce9643-947b-4c9e-a01d-5f00342ebcd9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:06.579545 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 514/1052] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:57:06 2022] GET /identity/v3/role_assignments?user.id=747829c276f9498989a49a1324c936a7&scope.project.id=0f7ee50e24ca44fb9ebded4e154bb2f1&role.id=58f91e3a13664dabb18c6227a1658fa6 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:57:06.582412 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c5587-7244-42f4-9ba2-f0c1ee2088f7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:06.582887 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c5587-7244-42f4-9ba2-f0c1ee2088f7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:06.583167 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c5587-7244-42f4-9ba2-f0c1ee2088f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:06.602786 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-075c5587-7244-42f4-9ba2-f0c1ee2088f7 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:57:06.604145 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-075c5587-7244-42f4-9ba2-f0c1ee2088f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:57:06.634068 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:06.651215 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfefef75-349d-4cb5-b5fd-985fa3cbaa89 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:06.682072 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 515/1053] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:57:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:57:06.699286 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58597dba-0022-40bd-9d04-78d2b4efe082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:06.699936 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58597dba-0022-40bd-9d04-78d2b4efe082 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:06.700222 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58597dba-0022-40bd-9d04-78d2b4efe082 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:06.700528 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58597dba-0022-40bd-9d04-78d2b4efe082 None admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:06.701321 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58597dba-0022-40bd-9d04-78d2b4efe082 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:06.701630 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58597dba-0022-40bd-9d04-78d2b4efe082 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:06.703093 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58597dba-0022-40bd-9d04-78d2b4efe082 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:06.705166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c5587-7244-42f4-9ba2-f0c1ee2088f7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:06.717866 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 539/1054] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 13:57:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:57:06.724093 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58597dba-0022-40bd-9d04-78d2b4efe082 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:06.778804 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:06.820193 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c5587-7244-42f4-9ba2-f0c1ee2088f7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:06.844376 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58597dba-0022-40bd-9d04-78d2b4efe082 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:06.858690 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb2774-442e-4f81-a102-1d509017e8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:06.859347 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb2774-442e-4f81-a102-1d509017e8f3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:06.859635 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb2774-442e-4f81-a102-1d509017e8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:06.859914 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb2774-442e-4f81-a102-1d509017e8f3 None admin] PATH_INFO: `/v3/services/image` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:06.860577 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeb2774-442e-4f81-a102-1d509017e8f3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:06.862069 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeb2774-442e-4f81-a102-1d509017e8f3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:06.871050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-faeb2774-442e-4f81-a102-1d509017e8f3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 13:57:06.871050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:06.871050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:57:06.871050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:57:06.871050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:57:06.871050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:57:06.871050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:57:06.871050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:57:06.871050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 13:57:06.871050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:57:06.871050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:57:06.871050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:57:06.871050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:06.871050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:06.871050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:06.871050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:06.871050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:06.872053 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:06.872053 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:06.872053 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:06.872053 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:06.872053 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:06.872053 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:06.872053 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:57:06.872053 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:57:06.872053 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:57:06.872053 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:57:06.872053 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:06.872053 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:06.872053 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:06.872053 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:06.872053 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:06.872053 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:06.873097 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:06.873097 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:06.873097 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:06.873097 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:06.873097 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:06.873097 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:06.873097 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:06.873097 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:06.873097 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:06.873097 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:06.873097 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:57:06.873097 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:57:06.873097 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 13:57:06.873097 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:57:06.873097 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 516/1055] 10.222.0.48 () {56 vars in 1105 bytes} [Mon Aug 1 13:57:06 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:57:06.878003 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb2774-442e-4f81-a102-1d509017e8f3 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:06.884114 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56773505-3529-46d4-98f7-eee6be5c6a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:06.884778 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56773505-3529-46d4-98f7-eee6be5c6a02 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:06.885065 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56773505-3529-46d4-98f7-eee6be5c6a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:06.885349 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56773505-3529-46d4-98f7-eee6be5c6a02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:06.886147 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56773505-3529-46d4-98f7-eee6be5c6a02 None admin] RBAC: Adding query filter params (role.id=05973988fe294b63969752e9dc8da875, scope.project.id=72ecec9fad964b1fbeaefd0519597229, user.id=a246654c783b4cc68d212e91bb05d8c6) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:06.886465 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56773505-3529-46d4-98f7-eee6be5c6a02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:06.887896 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56773505-3529-46d4-98f7-eee6be5c6a02 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:06.895006 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 540/1056] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:57:06 2022] GET /identity/v3/role_assignments?user.id=a246654c783b4cc68d212e91bb05d8c6&scope.project.id=72ecec9fad964b1fbeaefd0519597229&role.id=05973988fe294b63969752e9dc8da875 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:57:06.911639 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:06.994463 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb2774-442e-4f81-a102-1d509017e8f3 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:07.033138 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334f46d-659e-4d62-afcc-9871d7fda7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:07.033703 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334f46d-659e-4d62-afcc-9871d7fda7f3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:07.033989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334f46d-659e-4d62-afcc-9871d7fda7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:07.034269 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334f46d-659e-4d62-afcc-9871d7fda7f3 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:07.035564 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a334f46d-659e-4d62-afcc-9871d7fda7f3 None admin] RBAC: Adding query filter params (name=image) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:07.035877 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a334f46d-659e-4d62-afcc-9871d7fda7f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:07.037275 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a334f46d-659e-4d62-afcc-9871d7fda7f3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:07.042767 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 517/1057] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:57:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:57:07.048308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56773505-3529-46d4-98f7-eee6be5c6a02 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:07.154132 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56773505-3529-46d4-98f7-eee6be5c6a02 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:07.164538 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:07.189827 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ddf124-2cea-4b6f-9abb-8599f67cb021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:07.191171 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ddf124-2cea-4b6f-9abb-8599f67cb021 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:07.191602 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ddf124-2cea-4b6f-9abb-8599f67cb021 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:07.192046 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ddf124-2cea-4b6f-9abb-8599f67cb021 None admin] PATH_INFO: `/v3/services` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:07.192920 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ddf124-2cea-4b6f-9abb-8599f67cb021 None admin] RBAC: Adding query filter params (type=image) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:07.193373 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ddf124-2cea-4b6f-9abb-8599f67cb021 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:07.194888 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ddf124-2cea-4b6f-9abb-8599f67cb021 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:07.200624 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 541/1058] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:57:07 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:57:07.206128 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334f46d-659e-4d62-afcc-9871d7fda7f3 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:07.300917 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:07.311564 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334f46d-659e-4d62-afcc-9871d7fda7f3 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:07.348176 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a087b9-22e1-491a-8cbd-9db15910798f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:07.348823 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a087b9-22e1-491a-8cbd-9db15910798f None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:07.349109 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a087b9-22e1-491a-8cbd-9db15910798f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:07.349384 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a087b9-22e1-491a-8cbd-9db15910798f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:07.350090 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a087b9-22e1-491a-8cbd-9db15910798f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:07.351450 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a087b9-22e1-491a-8cbd-9db15910798f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:07.371160 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 518/1059] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 13:57:07 2022] POST /identity/v3/endpoints => generated 327 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 13:57:08.555764 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a6e34-5d80-45b2-9fe0-2b6cb4e01493 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:08.558458 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a6e34-5d80-45b2-9fe0-2b6cb4e01493 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:08.558458 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a6e34-5d80-45b2-9fe0-2b6cb4e01493 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:08.558458 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 542/1060] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:57:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:57:08.563070 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d7ea9-5f56-4af1-b022-75a210b8e521 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:08.563545 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d7ea9-5f56-4af1-b022-75a210b8e521 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:08.563706 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d7ea9-5f56-4af1-b022-75a210b8e521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:08.581870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-571d7ea9-5f56-4af1-b022-75a210b8e521 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:57:08.583043 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-571d7ea9-5f56-4af1-b022-75a210b8e521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:57:08.656362 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 519/1061] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:57:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:57:08.678808 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330a6e34-5d80-45b2-9fe0-2b6cb4e01493 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:08.780699 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330a6e34-5d80-45b2-9fe0-2b6cb4e01493 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:08.823139 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ebe64-14a6-4246-9864-cf10a7b16479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:08.823792 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ebe64-14a6-4246-9864-cf10a7b16479 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:08.824074 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ebe64-14a6-4246-9864-cf10a7b16479 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:08.824346 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ebe64-14a6-4246-9864-cf10a7b16479 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:08.835270 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2ebe64-14a6-4246-9864-cf10a7b16479 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:57:08.840265 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-5e2ebe64-14a6-4246-9864-cf10a7b16479 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:57:08.840265 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:08.840265 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:08.840265 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:08.840265 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:08.840265 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:08.840265 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:08.840265 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:08.840265 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:08.840265 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:08.840265 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:08.840265 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:08.840265 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:57:08.840265 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:57:08.840265 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:57:08.840265 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:57:08.841271 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:57:08.841271 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:57:08.841271 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:57:08.841271 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:57:08.841271 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:08.841271 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:08.841271 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:08.841271 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:08.841271 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:08.841271 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:08.841271 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:08.841271 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:08.841271 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:08.841271 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:08.841271 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:08.841271 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:08.847586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:08.847586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:08.847586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:08.847586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:08.847586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:57:08.847586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:57:08.847586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:57:08.847586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:57:08.847586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:57:08.847586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:57:08.847586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:57:08.847586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:57:08.847586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 543/1062] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 13:57:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:08.848312 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d7ea9-5f56-4af1-b022-75a210b8e521 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:08.900198 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3fc7e-0deb-4534-8504-65191d67e80b None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:08.900578 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3fc7e-0deb-4534-8504-65191d67e80b None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:08.900800 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3fc7e-0deb-4534-8504-65191d67e80b None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:08.903307 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 544/1063] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:57:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:57:08.908609 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1bae8-eef4-44b0-92bc-9ec7c3942c02 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:08.908900 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1bae8-eef4-44b0-92bc-9ec7c3942c02 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:08.909179 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1bae8-eef4-44b0-92bc-9ec7c3942c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:08.929483 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-cca1bae8-eef4-44b0-92bc-9ec7c3942c02 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:57:08.930712 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-cca1bae8-eef4-44b0-92bc-9ec7c3942c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:57:08.930830 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:08.959651 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:08.967034 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d7ea9-5f56-4af1-b022-75a210b8e521 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:09.000090 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 545/1064] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:57:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:57:09.007757 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b689534-64e5-4d87-8b3b-1827f69f4612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:09.008385 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b689534-64e5-4d87-8b3b-1827f69f4612 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:09.008698 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b689534-64e5-4d87-8b3b-1827f69f4612 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:09.008981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b689534-64e5-4d87-8b3b-1827f69f4612 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:09.009700 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b689534-64e5-4d87-8b3b-1827f69f4612 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:09.010009 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b689534-64e5-4d87-8b3b-1827f69f4612 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:09.011368 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b689534-64e5-4d87-8b3b-1827f69f4612 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:09.025185 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 520/1065] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:57:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:57:09.025382 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca1bae8-eef4-44b0-92bc-9ec7c3942c02 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:09.025915 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b689534-64e5-4d87-8b3b-1827f69f4612 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:09.124721 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca1bae8-eef4-44b0-92bc-9ec7c3942c02 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:09.154547 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b689534-64e5-4d87-8b3b-1827f69f4612 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:09.163659 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89d2d7-2012-4a58-8b50-477968292ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:09.164305 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89d2d7-2012-4a58-8b50-477968292ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:09.164620 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89d2d7-2012-4a58-8b50-477968292ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:09.164898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89d2d7-2012-4a58-8b50-477968292ea9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:09.175469 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b89d2d7-2012-4a58-8b50-477968292ea9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:57:09.180543 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-8b89d2d7-2012-4a58-8b50-477968292ea9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 13:57:09.180543 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:09.180543 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:09.180543 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:09.180543 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:09.180543 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:09.180543 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:09.180543 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:09.180543 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:09.180543 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:09.180543 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:09.180543 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:09.180543 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:57:09.180543 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:57:09.180543 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:57:09.180543 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:57:09.181450 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:57:09.181450 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:57:09.181450 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:57:09.181450 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:57:09.181450 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:09.181450 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:09.181450 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:09.181450 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:09.181450 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:09.181450 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:09.181450 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:09.181450 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:09.181450 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:09.181450 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:09.181450 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:09.181450 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:09.182274 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:09.182274 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:09.182274 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:09.182274 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:09.182274 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:57:09.182274 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:57:09.182274 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:57:09.182274 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:57:09.182274 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:57:09.182274 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:57:09.182274 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 13:57:09.182274 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:57:09.182274 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 546/1066] 10.222.0.48 () {56 vars in 1120 bytes} [Mon Aug 1 13:57:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:09.187539 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89d2d7-2012-4a58-8b50-477968292ea9 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:09.195630 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e8f798-a8b7-44ca-81e3-fd08313a753e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:09.196857 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8f798-a8b7-44ca-81e3-fd08313a753e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:09.196857 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8f798-a8b7-44ca-81e3-fd08313a753e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:09.197084 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8f798-a8b7-44ca-81e3-fd08313a753e None admin] PATH_INFO: `/v3/users/747829c276f9498989a49a1324c936a7` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:09.198931 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e8f798-a8b7-44ca-81e3-fd08313a753e None admin] RBAC: Authorizing `identity:get_user(user_id=747829c276f9498989a49a1324c936a7)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:09.200281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e8f798-a8b7-44ca-81e3-fd08313a753e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:09.202124 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 521/1067] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:57:09 2022] GET /identity/v3/users/747829c276f9498989a49a1324c936a7 => generated 285 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:57:09.208896 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e8f798-a8b7-44ca-81e3-fd08313a753e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:09.211905 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:09.218319 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:09.306000 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89d2d7-2012-4a58-8b50-477968292ea9 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:09.337448 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e8f798-a8b7-44ca-81e3-fd08313a753e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:09.345992 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdec631-7589-4a98-bceb-1eb082c294e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:09.346613 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdec631-7589-4a98-bceb-1eb082c294e5 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:09.346901 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdec631-7589-4a98-bceb-1eb082c294e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:09.347173 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdec631-7589-4a98-bceb-1eb082c294e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:09.347895 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdec631-7589-4a98-bceb-1eb082c294e5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:09.348198 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdec631-7589-4a98-bceb-1eb082c294e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:09.349599 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdec631-7589-4a98-bceb-1eb082c294e5 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:09.357615 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 547/1068] 10.222.0.48 () {56 vars in 1115 bytes} [Mon Aug 1 13:57:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:57:09.363479 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdec631-7589-4a98-bceb-1eb082c294e5 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:09.382204 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a138bdeb-1002-4595-82d9-521235ab11ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:09.383215 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138bdeb-1002-4595-82d9-521235ab11ae None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:09.383681 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138bdeb-1002-4595-82d9-521235ab11ae None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:09.384116 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138bdeb-1002-4595-82d9-521235ab11ae None admin] PATH_INFO: `/v3/projects/0f7ee50e24ca44fb9ebded4e154bb2f1` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:09.385649 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a138bdeb-1002-4595-82d9-521235ab11ae None admin] RBAC: Authorizing `identity:get_project(project_id=0f7ee50e24ca44fb9ebded4e154bb2f1)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:09.387223 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a138bdeb-1002-4595-82d9-521235ab11ae None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:09.389602 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 522/1069] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:57:09 2022] GET /identity/v3/projects/0f7ee50e24ca44fb9ebded4e154bb2f1 => generated 311 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:57:09.395699 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a138bdeb-1002-4595-82d9-521235ab11ae None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:09.454017 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:09.465367 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:09.469904 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdec631-7589-4a98-bceb-1eb082c294e5 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:09.497125 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a138bdeb-1002-4595-82d9-521235ab11ae None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:09.503483 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb98517-7b58-4381-8421-895e59539bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:09.504386 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb98517-7b58-4381-8421-895e59539bcd None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:09.504708 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb98517-7b58-4381-8421-895e59539bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:09.504989 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb98517-7b58-4381-8421-895e59539bcd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:09.515487 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb98517-7b58-4381-8421-895e59539bcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:09.516982 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb98517-7b58-4381-8421-895e59539bcd None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:09.531526 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90d25e-5260-44ea-a9ed-fc0e6947150b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:09.531672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-abb98517-7b58-4381-8421-895e59539bcd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:57:09.531672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:09.531672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:09.531672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:09.531672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:09.531672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:09.531672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:09.531672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:09.531672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:09.531672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:09.531672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:09.531672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:09.531672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:57:09.531672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:57:09.531672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:57:09.532591 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:57:09.532591 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:09.532591 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:09.532591 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:09.532591 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:09.532591 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:09.532591 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:09.532591 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:09.532591 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:09.532591 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:09.532591 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:09.532591 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:09.532591 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:09.532591 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:09.532591 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:09.532591 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:09.532591 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:09.533479 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:57:09.533479 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:57:09.533479 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:57:09.533479 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:57:09.533716 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90d25e-5260-44ea-a9ed-fc0e6947150b None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:09.533716 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90d25e-5260-44ea-a9ed-fc0e6947150b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:09.533866 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 548/1070] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:57:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:57:09.533942 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90d25e-5260-44ea-a9ed-fc0e6947150b None admin] PATH_INFO: `/v3/projects/0f7ee50e24ca44fb9ebded4e154bb2f1/users/747829c276f9498989a49a1324c936a7/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:09.534012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9d9eb-4938-4495-877a-be4031d9c596 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:09.534578 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9d9eb-4938-4495-877a-be4031d9c596 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:09.534578 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9d9eb-4938-4495-877a-be4031d9c596 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:09.535806 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 549/1071] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:57:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:57:09.536090 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb90d25e-5260-44ea-a9ed-fc0e6947150b None admin] RBAC: Authorizing `identity:create_grant(project_id=0f7ee50e24ca44fb9ebded4e154bb2f1, user_id=747829c276f9498989a49a1324c936a7, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:09.537601 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb90d25e-5260-44ea-a9ed-fc0e6947150b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:09.538762 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba9d9eb-4938-4495-877a-be4031d9c596 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:09.551873 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 523/1072] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 13:57:09 2022] PUT /identity/v3/projects/0f7ee50e24ca44fb9ebded4e154bb2f1/users/747829c276f9498989a49a1324c936a7/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:57:09.564792 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de741a0-9427-4381-8e37-b2b5734d74b4 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:09.564792 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de741a0-9427-4381-8e37-b2b5734d74b4 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:09.564792 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de741a0-9427-4381-8e37-b2b5734d74b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:09.585000 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-6de741a0-9427-4381-8e37-b2b5734d74b4 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:57:09.586235 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6de741a0-9427-4381-8e37-b2b5734d74b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:57:09.654498 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba9d9eb-4938-4495-877a-be4031d9c596 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:09.660913 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 524/1073] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:57:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:57:09.683534 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de741a0-9427-4381-8e37-b2b5734d74b4 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:09.692750 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5cd1a0-c9ec-40e3-acb0-c4aa5bc130c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:09.693394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5cd1a0-c9ec-40e3-acb0-c4aa5bc130c7 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:09.693678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5cd1a0-c9ec-40e3-acb0-c4aa5bc130c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:09.693978 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5cd1a0-c9ec-40e3-acb0-c4aa5bc130c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:09.694731 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5cd1a0-c9ec-40e3-acb0-c4aa5bc130c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:09.695106 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5cd1a0-c9ec-40e3-acb0-c4aa5bc130c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:09.696434 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5cd1a0-c9ec-40e3-acb0-c4aa5bc130c7 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:09.703955 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:09.714648 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 550/1074] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:57:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:57:09.721520 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5cd1a0-c9ec-40e3-acb0-c4aa5bc130c7 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:09.758759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:09.808693 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de741a0-9427-4381-8e37-b2b5734d74b4 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:09.837145 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5cd1a0-c9ec-40e3-acb0-c4aa5bc130c7 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:09.847930 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166ab07-5001-4eb9-be6b-4f1936b77063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:09.848768 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166ab07-5001-4eb9-be6b-4f1936b77063 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:09.848905 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166ab07-5001-4eb9-be6b-4f1936b77063 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:09.849126 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166ab07-5001-4eb9-be6b-4f1936b77063 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:09.874348 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c987a19a-a51d-4657-a814-abe0c98fcd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:09.874997 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c987a19a-a51d-4657-a814-abe0c98fcd1f None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:09.875283 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c987a19a-a51d-4657-a814-abe0c98fcd1f None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:09.875562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c987a19a-a51d-4657-a814-abe0c98fcd1f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:09.884120 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c987a19a-a51d-4657-a814-abe0c98fcd1f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:09.885605 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c987a19a-a51d-4657-a814-abe0c98fcd1f None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:09.898710 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-c987a19a-a51d-4657-a814-abe0c98fcd1f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 13:57:09.898710 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:09.898710 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:09.898710 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:09.898710 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:09.898710 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:09.898710 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:09.898710 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:09.898710 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:09.898710 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:09.898710 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:09.898710 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:09.898710 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:57:09.898710 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:57:09.898710 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:57:09.898710 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:57:09.899730 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:09.899730 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:09.899730 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:57:09.899730 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:57:09.899730 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:57:09.899730 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:57:09.899730 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:09.899730 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:09.899730 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:09.899730 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:09.899730 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:09.899730 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:09.899730 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:09.899730 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:09.899730 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:09.899730 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:09.899730 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:09.899730 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:09.900698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:09.900698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:09.900698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:57:09.900698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:57:09.900698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:57:09.900698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:57:09.900698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:57:09.900698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:57:09.900698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 13:57:09.900698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:57:09.904725 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 551/1075] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Aug 1 13:57:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:09.905445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c987a19a-a51d-4657-a814-abe0c98fcd1f None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:09.924458 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f166ab07-5001-4eb9-be6b-4f1936b77063 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:09.926565 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f166ab07-5001-4eb9-be6b-4f1936b77063 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:09.966172 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:09.990220 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:10.011187 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c987a19a-a51d-4657-a814-abe0c98fcd1f None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:10.034061 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 525/1076] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:57:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 13:57:10.040118 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166ab07-5001-4eb9-be6b-4f1936b77063 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:10.048200 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7e6ee-6953-40ce-88ce-b2304a07d59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:10.048848 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7e6ee-6953-40ce-88ce-b2304a07d59a None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:10.049134 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7e6ee-6953-40ce-88ce-b2304a07d59a None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:10.049400 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7e6ee-6953-40ce-88ce-b2304a07d59a None admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:10.050766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c7e6ee-6953-40ce-88ce-b2304a07d59a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:10.051071 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c7e6ee-6953-40ce-88ce-b2304a07d59a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:10.052511 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c7e6ee-6953-40ce-88ce-b2304a07d59a None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:10.070336 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 552/1077] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Aug 1 13:57:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:57:10.076474 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7e6ee-6953-40ce-88ce-b2304a07d59a None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:10.151756 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166ab07-5001-4eb9-be6b-4f1936b77063 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:10.182290 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7e6ee-6953-40ce-88ce-b2304a07d59a None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:10.190035 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5800812-8161-4139-a677-06a1842d1b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:10.190723 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5800812-8161-4139-a677-06a1842d1b57 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:10.191008 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5800812-8161-4139-a677-06a1842d1b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:10.191292 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5800812-8161-4139-a677-06a1842d1b57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:10.198775 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:10.204733 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5800812-8161-4139-a677-06a1842d1b57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:10.206232 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5800812-8161-4139-a677-06a1842d1b57 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:10.217955 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcea081-4023-414f-bc4d-b31ca1dcfba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:10.218595 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcea081-4023-414f-bc4d-b31ca1dcfba8 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:10.218878 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcea081-4023-414f-bc4d-b31ca1dcfba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:10.219155 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcea081-4023-414f-bc4d-b31ca1dcfba8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:10.221095 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-a5800812-8161-4139-a677-06a1842d1b57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:57:10.221095 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:10.221095 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:10.221095 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:10.221095 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:10.221095 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:10.221095 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:10.221095 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:10.221095 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:10.221095 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:10.221095 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:10.221095 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:10.221095 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:57:10.221095 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:57:10.221095 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:57:10.222530 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:57:10.222530 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:10.222530 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:10.222530 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:10.222530 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:10.222530 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:10.222530 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:10.222530 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:10.222530 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:10.222530 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:10.222530 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:10.222530 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:10.222530 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:10.222530 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:10.222530 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:10.222530 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:10.222530 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:10.223451 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:57:10.223451 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:57:10.223451 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:57:10.223451 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:57:10.223451 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 526/1078] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:57:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:57:10.228781 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5800812-8161-4139-a677-06a1842d1b57 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:10.232766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcea081-4023-414f-bc4d-b31ca1dcfba8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:10.234230 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcea081-4023-414f-bc4d-b31ca1dcfba8 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:10.244724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:10.251051 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-dbcea081-4023-414f-bc4d-b31ca1dcfba8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:57:10.251051 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:10.251051 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:10.251051 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:10.251051 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:10.251051 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:10.251051 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:10.251051 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:10.251051 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:10.251051 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:10.251051 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:10.251051 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:10.251051 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:57:10.251051 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:57:10.251051 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:57:10.251961 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:57:10.251961 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:10.251961 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:10.251961 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:10.251961 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:10.251961 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:10.251961 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:10.251961 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:10.251961 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:10.251961 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:10.251961 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:10.251961 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:10.251961 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:10.251961 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:10.251961 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:10.251961 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:10.251961 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:10.252930 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:57:10.252930 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:57:10.252930 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:57:10.252930 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:57:10.257330 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 553/1079] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:57:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:57:10.258168 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcea081-4023-414f-bc4d-b31ca1dcfba8 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:10.340544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5800812-8161-4139-a677-06a1842d1b57 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:10.369161 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcea081-4023-414f-bc4d-b31ca1dcfba8 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:10.375989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aac4a6-e3cd-4061-8650-f873b11cd3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:10.376649 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aac4a6-e3cd-4061-8650-f873b11cd3c6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:10.376938 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aac4a6-e3cd-4061-8650-f873b11cd3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:10.377218 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aac4a6-e3cd-4061-8650-f873b11cd3c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:10.377911 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aac4a6-e3cd-4061-8650-f873b11cd3c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:10.378251 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aac4a6-e3cd-4061-8650-f873b11cd3c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:10.379871 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aac4a6-e3cd-4061-8650-f873b11cd3c6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:10.391639 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 527/1080] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:57:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:57:10.417558 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0cec3b-b7ad-486e-880d-ac83c41a6c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:10.418212 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0cec3b-b7ad-486e-880d-ac83c41a6c8e None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:10.418495 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0cec3b-b7ad-486e-880d-ac83c41a6c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:10.418769 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0cec3b-b7ad-486e-880d-ac83c41a6c8e None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:10.419561 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0cec3b-b7ad-486e-880d-ac83c41a6c8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:10.419869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0cec3b-b7ad-486e-880d-ac83c41a6c8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:10.421383 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0cec3b-b7ad-486e-880d-ac83c41a6c8e None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:10.439530 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 554/1081] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:57:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:10.441324 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aac4a6-e3cd-4061-8650-f873b11cd3c6 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:10.517551 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:10.551360 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aac4a6-e3cd-4061-8650-f873b11cd3c6 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:10.591349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e9ac8a-dfc5-4742-be8f-8e0657064b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:10.592694 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9ac8a-dfc5-4742-be8f-8e0657064b5d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:10.592977 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9ac8a-dfc5-4742-be8f-8e0657064b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:10.593251 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9ac8a-dfc5-4742-be8f-8e0657064b5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:10.602251 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e9ac8a-dfc5-4742-be8f-8e0657064b5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:10.603736 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e9ac8a-dfc5-4742-be8f-8e0657064b5d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:10.621109 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-76e9ac8a-dfc5-4742-be8f-8e0657064b5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:57:10.621109 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:10.621109 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:10.621109 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:10.621109 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:10.621109 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:10.621109 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:10.621109 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:10.621109 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:10.621109 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:10.621109 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:10.621109 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:10.621109 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:57:10.621109 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:57:10.621109 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:57:10.622069 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:57:10.622069 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:10.622069 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:10.622069 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:10.622069 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:10.622069 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:10.622069 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:10.622069 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:10.622069 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:10.622069 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:10.622069 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:10.622069 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:10.622069 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:10.622069 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:10.622069 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:10.622069 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:10.622069 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:10.623170 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:57:10.623170 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:57:10.623170 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:57:10.623170 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:57:10.623170 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:57:10.623170 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:57:10.623170 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:57:10.623170 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:57:10.623170 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 528/1082] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:57:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:10.631402 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0cec3b-b7ad-486e-880d-ac83c41a6c8e None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:10.676604 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:10.756440 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0cec3b-b7ad-486e-880d-ac83c41a6c8e None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:10.790255 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274efe4e-f786-46a5-b610-53bafb0e5b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:10.790871 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274efe4e-f786-46a5-b610-53bafb0e5b9c None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:10.791158 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274efe4e-f786-46a5-b610-53bafb0e5b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:10.791443 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274efe4e-f786-46a5-b610-53bafb0e5b9c None admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:10.792191 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274efe4e-f786-46a5-b610-53bafb0e5b9c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:10.792523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274efe4e-f786-46a5-b610-53bafb0e5b9c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:10.793951 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274efe4e-f786-46a5-b610-53bafb0e5b9c None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:10.805524 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 555/1083] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 13:57:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:10.811446 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e9ac8a-dfc5-4742-be8f-8e0657064b5d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:10.908643 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e9ac8a-dfc5-4742-be8f-8e0657064b5d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:10.936861 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:10.942838 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12668500-4e32-4f4f-8dbc-26b7cd94f090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:10.943500 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12668500-4e32-4f4f-8dbc-26b7cd94f090 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:10.943784 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12668500-4e32-4f4f-8dbc-26b7cd94f090 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:10.944063 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12668500-4e32-4f4f-8dbc-26b7cd94f090 None admin] PATH_INFO: `/v3/projects/72ecec9fad964b1fbeaefd0519597229/users/a246654c783b4cc68d212e91bb05d8c6/roles/05973988fe294b63969752e9dc8da875` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:10.946217 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12668500-4e32-4f4f-8dbc-26b7cd94f090 None admin] RBAC: Authorizing `identity:create_grant(project_id=72ecec9fad964b1fbeaefd0519597229, user_id=a246654c783b4cc68d212e91bb05d8c6, role_id=05973988fe294b63969752e9dc8da875)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:10.947677 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12668500-4e32-4f4f-8dbc-26b7cd94f090 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:10.960830 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 529/1084] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 13:57:10 2022] PUT /identity/v3/projects/72ecec9fad964b1fbeaefd0519597229/users/a246654c783b4cc68d212e91bb05d8c6/roles/05973988fe294b63969752e9dc8da875 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 13:57:11.569857 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4708f-f73b-44d0-8db0-fa665f43e9df None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:11.570147 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4708f-f73b-44d0-8db0-fa665f43e9df None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:11.570429 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4708f-f73b-44d0-8db0-fa665f43e9df None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:11.571811 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 556/1085] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:57:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:57:11.576940 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40884c7a-2910-45e4-8e9d-caf4c34c2fdc None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:11.577304 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40884c7a-2910-45e4-8e9d-caf4c34c2fdc None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:11.577659 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40884c7a-2910-45e4-8e9d-caf4c34c2fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:11.595174 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-40884c7a-2910-45e4-8e9d-caf4c34c2fdc None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:57:11.596343 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-40884c7a-2910-45e4-8e9d-caf4c34c2fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:57:11.662533 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 530/1086] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:57:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:57:11.690871 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4708f-f73b-44d0-8db0-fa665f43e9df None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:11.776172 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:11.789666 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4708f-f73b-44d0-8db0-fa665f43e9df None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:11.825500 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5ad3eb-5f48-4c60-8bfa-65617a5926db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:11.826150 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ad3eb-5f48-4c60-8bfa-65617a5926db None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:11.826437 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ad3eb-5f48-4c60-8bfa-65617a5926db None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:11.826714 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ad3eb-5f48-4c60-8bfa-65617a5926db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:11.837014 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5ad3eb-5f48-4c60-8bfa-65617a5926db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:57:11.841903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-4e5ad3eb-5f48-4c60-8bfa-65617a5926db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:57:11.841903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:11.841903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:11.841903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:11.841903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:11.841903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:11.841903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:11.841903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:11.841903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:11.841903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:11.841903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:11.841903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:11.841903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:57:11.841903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:57:11.841903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:57:11.841903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:57:11.842869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:57:11.842869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:57:11.842869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:57:11.842869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:57:11.842869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:11.842869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:11.842869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:11.842869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:11.842869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:11.842869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:11.842869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:11.842869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:11.842869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:11.842869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:11.842869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:11.842869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:11.843696 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:11.843696 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:11.843696 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:11.843696 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:11.843696 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:57:11.843696 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:57:11.843696 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:57:11.843696 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:57:11.843696 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:57:11.843696 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:57:11.843696 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:57:11.843696 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:57:11.844790 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 557/1087] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 13:57:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:11.849572 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40884c7a-2910-45e4-8e9d-caf4c34c2fdc None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:11.947219 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40884c7a-2910-45e4-8e9d-caf4c34c2fdc None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:11.968994 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:11.981479 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832cbf37-0a80-4efc-aef5-a2d1aea56100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:11.982100 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cbf37-0a80-4efc-aef5-a2d1aea56100 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:11.982389 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cbf37-0a80-4efc-aef5-a2d1aea56100 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:11.982672 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cbf37-0a80-4efc-aef5-a2d1aea56100 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:11.983401 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832cbf37-0a80-4efc-aef5-a2d1aea56100 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:11.983713 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832cbf37-0a80-4efc-aef5-a2d1aea56100 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:11.985105 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832cbf37-0a80-4efc-aef5-a2d1aea56100 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:11.993913 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 531/1088] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:57:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:11.999398 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5ad3eb-5f48-4c60-8bfa-65617a5926db None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:12.098502 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5ad3eb-5f48-4c60-8bfa-65617a5926db None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:12.131477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a755877-2ab6-4a7b-ad10-2defe621bf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:12.132110 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a755877-2ab6-4a7b-ad10-2defe621bf7f None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:12.132391 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a755877-2ab6-4a7b-ad10-2defe621bf7f None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:12.132693 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a755877-2ab6-4a7b-ad10-2defe621bf7f None admin] PATH_INFO: `/v3/users/747829c276f9498989a49a1324c936a7` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:12.133840 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a755877-2ab6-4a7b-ad10-2defe621bf7f None admin] RBAC: Authorizing `identity:get_user(user_id=747829c276f9498989a49a1324c936a7)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:12.135265 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a755877-2ab6-4a7b-ad10-2defe621bf7f None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:12.136871 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 558/1089] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:57:11 2022] GET /identity/v3/users/747829c276f9498989a49a1324c936a7 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:12.142665 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832cbf37-0a80-4efc-aef5-a2d1aea56100 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:12.238905 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832cbf37-0a80-4efc-aef5-a2d1aea56100 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:12.271161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c1233c-871c-4ab0-af90-4efca9781774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:12.271768 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1233c-871c-4ab0-af90-4efca9781774 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:12.272048 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1233c-871c-4ab0-af90-4efca9781774 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:12.272329 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1233c-871c-4ab0-af90-4efca9781774 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:12.323027 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:12.339116 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c1233c-871c-4ab0-af90-4efca9781774 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:12.340621 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c1233c-871c-4ab0-af90-4efca9781774 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:12.435904 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 532/1090] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:57:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 13:57:12.442559 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a755877-2ab6-4a7b-ad10-2defe621bf7f None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:12.461360 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:12.542677 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a755877-2ab6-4a7b-ad10-2defe621bf7f None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:12.575558 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f0362-7c15-4d42-a698-79d101f63ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:12.576197 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f0362-7c15-4d42-a698-79d101f63ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:12.576495 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f0362-7c15-4d42-a698-79d101f63ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:12.576775 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f0362-7c15-4d42-a698-79d101f63ac1 None admin] PATH_INFO: `/v3/projects/0f7ee50e24ca44fb9ebded4e154bb2f1` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:12.577913 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236f0362-7c15-4d42-a698-79d101f63ac1 None admin] RBAC: Authorizing `identity:get_project(project_id=0f7ee50e24ca44fb9ebded4e154bb2f1)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:12.579339 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236f0362-7c15-4d42-a698-79d101f63ac1 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:12.581603 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 559/1091] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:57:12 2022] GET /identity/v3/projects/0f7ee50e24ca44fb9ebded4e154bb2f1 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:12.587621 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c1233c-871c-4ab0-af90-4efca9781774 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:12.668959 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:12.685222 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c1233c-871c-4ab0-af90-4efca9781774 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:12.727929 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829a4442-8de8-4c4f-87c3-712bc106f4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:12.728460 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a4442-8de8-4c4f-87c3-712bc106f4f4 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:12.728773 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a4442-8de8-4c4f-87c3-712bc106f4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:12.729052 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a4442-8de8-4c4f-87c3-712bc106f4f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:12.729836 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829a4442-8de8-4c4f-87c3-712bc106f4f4 None admin] RBAC: Adding query filter params (role.id=58f91e3a13664dabb18c6227a1658fa6, scope.project.id=0f7ee50e24ca44fb9ebded4e154bb2f1, user.id=747829c276f9498989a49a1324c936a7) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:12.730147 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829a4442-8de8-4c4f-87c3-712bc106f4f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:12.731577 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829a4442-8de8-4c4f-87c3-712bc106f4f4 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:12.737766 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 533/1092] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:57:12 2022] GET /identity/v3/role_assignments?user.id=747829c276f9498989a49a1324c936a7&scope.project.id=0f7ee50e24ca44fb9ebded4e154bb2f1&role.id=58f91e3a13664dabb18c6227a1658fa6 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:12.949797 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f2538-f0fd-453c-a510-e5dfc3af861e None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:12.950309 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f2538-f0fd-453c-a510-e5dfc3af861e None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:12.950424 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f2538-f0fd-453c-a510-e5dfc3af861e None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:12.951702 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 560/1093] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:57:12.956844 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24620d5-a6eb-4289-a6b5-2c3664a3fa4b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:12.957130 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24620d5-a6eb-4289-a6b5-2c3664a3fa4b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:12.957405 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24620d5-a6eb-4289-a6b5-2c3664a3fa4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:12.974881 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c24620d5-a6eb-4289-a6b5-2c3664a3fa4b None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:57:12.976043 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c24620d5-a6eb-4289-a6b5-2c3664a3fa4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:57:13.041293 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 534/1094] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:57:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:57:13.064603 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f2538-f0fd-453c-a510-e5dfc3af861e None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:13.153470 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:13.173434 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f2538-f0fd-453c-a510-e5dfc3af861e None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:13.206903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945b0e5-46ba-4c9c-94da-2160a2339dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:13.207547 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945b0e5-46ba-4c9c-94da-2160a2339dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:13.207826 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945b0e5-46ba-4c9c-94da-2160a2339dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:13.208100 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945b0e5-46ba-4c9c-94da-2160a2339dc1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:13.218343 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a945b0e5-46ba-4c9c-94da-2160a2339dc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:57:13.223650 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-a945b0e5-46ba-4c9c-94da-2160a2339dc1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 13:57:13.223650 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:13.223650 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:13.223650 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:13.223650 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:13.223650 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:13.223650 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:13.223650 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:13.223650 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:13.223650 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:13.223650 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:13.223650 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:13.223650 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:57:13.223650 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:57:13.223650 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:57:13.223650 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:57:13.224800 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:57:13.224800 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:57:13.224800 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:57:13.224800 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:57:13.224800 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:13.224800 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:13.224800 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:13.224800 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:13.224800 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:13.224800 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:13.224800 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:13.224800 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:13.224800 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:13.224800 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:13.224800 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:13.224800 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:13.225668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:13.225668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:13.225668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:13.225668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:13.225668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:57:13.225668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:57:13.225668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:57:13.225668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:57:13.225668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:57:13.225668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:57:13.225668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 13:57:13.225668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:57:13.225668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 561/1095] 10.222.0.48 () {56 vars in 1120 bytes} [Mon Aug 1 13:57:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:13.230872 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24620d5-a6eb-4289-a6b5-2c3664a3fa4b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:13.308539 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:13.330104 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24620d5-a6eb-4289-a6b5-2c3664a3fa4b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:13.363370 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0c0134-6768-450e-85c8-be019c806253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:13.363996 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c0134-6768-450e-85c8-be019c806253 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:13.364283 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c0134-6768-450e-85c8-be019c806253 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:13.364587 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c0134-6768-450e-85c8-be019c806253 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:13.365307 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0c0134-6768-450e-85c8-be019c806253 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:13.365615 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0c0134-6768-450e-85c8-be019c806253 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:13.366977 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0c0134-6768-450e-85c8-be019c806253 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:13.374845 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 535/1096] 10.222.0.48 () {56 vars in 1115 bytes} [Mon Aug 1 13:57:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:57:13.381385 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945b0e5-46ba-4c9c-94da-2160a2339dc1 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:13.481501 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945b0e5-46ba-4c9c-94da-2160a2339dc1 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:13.515296 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292299eb-60e3-444c-b699-f53dfdb7f77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:13.515932 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292299eb-60e3-444c-b699-f53dfdb7f77e None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:13.516211 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292299eb-60e3-444c-b699-f53dfdb7f77e None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:13.516506 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292299eb-60e3-444c-b699-f53dfdb7f77e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:13.526758 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292299eb-60e3-444c-b699-f53dfdb7f77e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:13.528201 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292299eb-60e3-444c-b699-f53dfdb7f77e None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:13.530646 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:13.542810 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-292299eb-60e3-444c-b699-f53dfdb7f77e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:57:13.542810 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:13.542810 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:13.542810 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:13.542810 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:13.542810 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:13.542810 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:13.542810 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:13.542810 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:13.542810 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:13.542810 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:13.542810 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:13.542810 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:57:13.542810 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:57:13.542810 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:57:13.543835 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:57:13.543835 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:13.543835 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:13.543835 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:13.543835 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:13.543835 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:13.543835 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:13.543835 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:13.543835 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:13.543835 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:13.543835 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:13.543835 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:13.543835 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:13.543835 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:13.543835 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:13.543835 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:13.543835 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:13.544765 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:57:13.544765 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:57:13.544765 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:57:13.544765 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:57:13.544765 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 562/1097] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:57:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:13.550191 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0c0134-6768-450e-85c8-be019c806253 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:13.645341 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0c0134-6768-450e-85c8-be019c806253 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:13.677484 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2858f4-1ab6-4871-91a6-ae0a0364f4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:13.678087 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2858f4-1ab6-4871-91a6-ae0a0364f4c0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:13.678368 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2858f4-1ab6-4871-91a6-ae0a0364f4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:13.678641 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2858f4-1ab6-4871-91a6-ae0a0364f4c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:13.679327 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2858f4-1ab6-4871-91a6-ae0a0364f4c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:13.679631 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2858f4-1ab6-4871-91a6-ae0a0364f4c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:13.681015 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2858f4-1ab6-4871-91a6-ae0a0364f4c0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:13.689457 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:13.696785 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 536/1098] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:57:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:13.698773 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292299eb-60e3-444c-b699-f53dfdb7f77e None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:13.795733 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292299eb-60e3-444c-b699-f53dfdb7f77e None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:13.832430 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664c493b-a330-49d8-917c-b4c66653373d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:13.833132 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c493b-a330-49d8-917c-b4c66653373d None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:13.833418 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c493b-a330-49d8-917c-b4c66653373d None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:13.833699 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c493b-a330-49d8-917c-b4c66653373d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:13.841135 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664c493b-a330-49d8-917c-b4c66653373d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:13.842621 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664c493b-a330-49d8-917c-b4c66653373d None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:13.854380 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-664c493b-a330-49d8-917c-b4c66653373d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 13:57:13.854380 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:13.854380 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:13.854380 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:13.854380 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:13.854380 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:13.854380 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:13.854380 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:13.854380 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:13.854380 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:13.854380 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:13.854380 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:13.854380 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:57:13.854380 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:57:13.854380 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:57:13.854380 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:57:13.855475 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:13.855475 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:13.855475 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:57:13.855475 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:57:13.855475 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:57:13.855475 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:57:13.855475 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:13.855475 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:13.855475 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:13.855475 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:13.855475 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:13.855475 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:13.855475 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:13.855475 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:13.855475 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:13.855475 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:13.855475 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:13.855475 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:13.861348 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:13.861348 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:13.861348 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:57:13.861348 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:57:13.861348 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:57:13.861348 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:57:13.861348 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:57:13.861348 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:57:13.861348 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 13:57:13.861348 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:57:13.861348 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 563/1099] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Aug 1 13:57:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:13.862698 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2858f4-1ab6-4871-91a6-ae0a0364f4c0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:13.959283 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2858f4-1ab6-4871-91a6-ae0a0364f4c0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:13.992462 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13225b30-c989-47e3-8e34-8414ef2d6f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:13.993100 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13225b30-c989-47e3-8e34-8414ef2d6f87 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:13.993383 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13225b30-c989-47e3-8e34-8414ef2d6f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:13.993658 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13225b30-c989-47e3-8e34-8414ef2d6f87 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:13.994442 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13225b30-c989-47e3-8e34-8414ef2d6f87 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:13.994751 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13225b30-c989-47e3-8e34-8414ef2d6f87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:13.996177 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13225b30-c989-47e3-8e34-8414ef2d6f87 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:14.012816 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 537/1100] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Aug 1 13:57:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:14.018553 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664c493b-a330-49d8-917c-b4c66653373d None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:14.073819 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:14.117647 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664c493b-a330-49d8-917c-b4c66653373d None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:14.150610 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65682f9f-ee3b-45a9-be88-c14172b9ee50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:14.151222 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65682f9f-ee3b-45a9-be88-c14172b9ee50 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:14.151503 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65682f9f-ee3b-45a9-be88-c14172b9ee50 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:14.151778 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65682f9f-ee3b-45a9-be88-c14172b9ee50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:14.216563 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65682f9f-ee3b-45a9-be88-c14172b9ee50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:14.217936 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65682f9f-ee3b-45a9-be88-c14172b9ee50 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:14.271479 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:14.314823 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 564/1101] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:57:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 13:57:14.322134 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13225b30-c989-47e3-8e34-8414ef2d6f87 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:14.381153 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:14.424008 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13225b30-c989-47e3-8e34-8414ef2d6f87 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:14.462227 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb26e5-9076-4413-b201-fb5139992309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:14.462885 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb26e5-9076-4413-b201-fb5139992309 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:14.463164 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb26e5-9076-4413-b201-fb5139992309 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:14.463439 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb26e5-9076-4413-b201-fb5139992309 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:14.477243 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fb26e5-9076-4413-b201-fb5139992309 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:14.478624 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fb26e5-9076-4413-b201-fb5139992309 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:14.493708 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-a7fb26e5-9076-4413-b201-fb5139992309 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:57:14.493708 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:14.493708 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:14.493708 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:14.493708 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:14.493708 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:14.493708 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:14.493708 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:14.493708 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:14.493708 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:14.493708 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:14.493708 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:14.493708 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:57:14.493708 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:57:14.493708 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:57:14.494581 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:57:14.494581 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:14.494581 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:14.494581 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:14.494581 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:14.494581 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:14.494581 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:14.494581 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:14.494581 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:14.494581 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:14.494581 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:14.494581 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:14.494581 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:14.494581 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:14.494581 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:14.494581 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:14.494581 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:14.495481 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:57:14.495481 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:57:14.495481 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:57:14.495481 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:57:14.495481 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 538/1102] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:57:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:14.500567 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65682f9f-ee3b-45a9-be88-c14172b9ee50 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:14.597463 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65682f9f-ee3b-45a9-be88-c14172b9ee50 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:14.630496 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14526ab-0d85-45e5-af54-f9d3c7ec1554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:14.631126 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14526ab-0d85-45e5-af54-f9d3c7ec1554 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:14.631409 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14526ab-0d85-45e5-af54-f9d3c7ec1554 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:14.631687 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14526ab-0d85-45e5-af54-f9d3c7ec1554 None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:14.632381 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14526ab-0d85-45e5-af54-f9d3c7ec1554 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:14.632715 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14526ab-0d85-45e5-af54-f9d3c7ec1554 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:14.634076 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14526ab-0d85-45e5-af54-f9d3c7ec1554 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:14.645223 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 565/1103] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:57:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:14.650653 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb26e5-9076-4413-b201-fb5139992309 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:14.749426 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb26e5-9076-4413-b201-fb5139992309 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:14.759780 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:14.779453 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b8bff-dd3a-4aab-bce9-8c2cb1af467a None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:14.779741 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b8bff-dd3a-4aab-bce9-8c2cb1af467a None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:14.780023 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b8bff-dd3a-4aab-bce9-8c2cb1af467a None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:14.781398 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 566/1104] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:57:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:57:14.784536 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d057bc-91a6-4884-864e-3fd24adc1e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:14.785195 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d057bc-91a6-4884-864e-3fd24adc1e4c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:14.785508 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d057bc-91a6-4884-864e-3fd24adc1e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:14.785791 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d057bc-91a6-4884-864e-3fd24adc1e4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:14.786825 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0679ecbf-8014-4604-9d15-403f1c39655a None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:14.787116 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0679ecbf-8014-4604-9d15-403f1c39655a None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:14.787398 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0679ecbf-8014-4604-9d15-403f1c39655a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:14.792879 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d057bc-91a6-4884-864e-3fd24adc1e4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:14.795504 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d057bc-91a6-4884-864e-3fd24adc1e4c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:14.807962 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-0679ecbf-8014-4604-9d15-403f1c39655a None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:57:14.810048 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-0679ecbf-8014-4604-9d15-403f1c39655a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:57:14.811310 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-11d057bc-91a6-4884-864e-3fd24adc1e4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:57:14.811310 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:14.811310 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:14.811310 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:14.811310 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:14.811310 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:14.811310 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:14.811310 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:14.811310 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:14.811310 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:14.811310 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:14.811310 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:14.811310 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:57:14.811310 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:57:14.811310 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:57:14.812216 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:57:14.812216 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:14.812216 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:14.812216 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:14.812216 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:14.812216 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:14.812216 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:14.812216 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:14.812216 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:14.812216 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:14.812216 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:14.812216 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:14.812216 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:14.812216 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:14.812216 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:14.812216 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:14.812216 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:14.814983 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:57:14.814983 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:57:14.814983 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:57:14.814983 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:57:14.814983 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:57:14.814983 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:57:14.814983 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:57:14.814983 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:57:14.814983 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 539/1105] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:57:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:14.819279 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d057bc-91a6-4884-864e-3fd24adc1e4c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:14.827930 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:14.887043 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 567/1106] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:57:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 13:57:14.912090 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0679ecbf-8014-4604-9d15-403f1c39655a None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:14.921622 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d057bc-91a6-4884-864e-3fd24adc1e4c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:14.954309 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5cbd5-3723-49a8-a23e-6101a5801726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:14.954916 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5cbd5-3723-49a8-a23e-6101a5801726 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:14.955195 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5cbd5-3723-49a8-a23e-6101a5801726 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:14.955467 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5cbd5-3723-49a8-a23e-6101a5801726 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:14.956763 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d5cbd5-3723-49a8-a23e-6101a5801726 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:14.957076 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d5cbd5-3723-49a8-a23e-6101a5801726 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:14.958481 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d5cbd5-3723-49a8-a23e-6101a5801726 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:14.970059 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 540/1107] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 13:57:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:57:14.976738 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5cbd5-3723-49a8-a23e-6101a5801726 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:15.005191 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:15.014268 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0679ecbf-8014-4604-9d15-403f1c39655a None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:15.047957 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a1ce1-d1b6-4000-9ee6-98b09f55a6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:15.048619 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a1ce1-d1b6-4000-9ee6-98b09f55a6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:15.048902 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a1ce1-d1b6-4000-9ee6-98b09f55a6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:15.049172 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a1ce1-d1b6-4000-9ee6-98b09f55a6f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:15.050311 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a1ce1-d1b6-4000-9ee6-98b09f55a6f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:15.051720 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a1ce1-d1b6-4000-9ee6-98b09f55a6f2 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:15.053326 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 568/1108] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:57:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:57:15.058880 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a1ce1-d1b6-4000-9ee6-98b09f55a6f2 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:15.074029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:15.080000 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5cbd5-3723-49a8-a23e-6101a5801726 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:15.113605 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee80e9-5823-48b8-808e-28c7e01d6e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:15.114204 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee80e9-5823-48b8-808e-28c7e01d6e4e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:15.114484 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee80e9-5823-48b8-808e-28c7e01d6e4e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:15.114755 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee80e9-5823-48b8-808e-28c7e01d6e4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:15.115534 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ee80e9-5823-48b8-808e-28c7e01d6e4e None admin] RBAC: Adding query filter params (role.id=05973988fe294b63969752e9dc8da875, scope.project.id=72ecec9fad964b1fbeaefd0519597229, user.id=a246654c783b4cc68d212e91bb05d8c6) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:15.115837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ee80e9-5823-48b8-808e-28c7e01d6e4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:15.117324 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ee80e9-5823-48b8-808e-28c7e01d6e4e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:15.123193 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 541/1109] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:57:14 2022] GET /identity/v3/role_assignments?user.id=a246654c783b4cc68d212e91bb05d8c6&scope.project.id=72ecec9fad964b1fbeaefd0519597229&role.id=05973988fe294b63969752e9dc8da875 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:15.161511 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a1ce1-d1b6-4000-9ee6-98b09f55a6f2 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:15.198640 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be6d79d-5ec3-42e0-96aa-a0e6bd1533ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:15.199276 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6d79d-5ec3-42e0-96aa-a0e6bd1533ae None admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:15.199560 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6d79d-5ec3-42e0-96aa-a0e6bd1533ae None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:15.199837 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6d79d-5ec3-42e0-96aa-a0e6bd1533ae None admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:15.200563 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be6d79d-5ec3-42e0-96aa-a0e6bd1533ae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:15.202017 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be6d79d-5ec3-42e0-96aa-a0e6bd1533ae None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:15.236513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 569/1110] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 13:57:15 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 13:57:17.245086 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86fd73-f4ef-456b-8871-dee261b1ae45 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:17.245608 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86fd73-f4ef-456b-8871-dee261b1ae45 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:17.245732 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86fd73-f4ef-456b-8871-dee261b1ae45 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:17.246994 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 542/1111] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:57:17.252057 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b54c5-0946-44ff-af09-6209a8dd9976 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:17.253143 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b54c5-0946-44ff-af09-6209a8dd9976 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:17.253235 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b54c5-0946-44ff-af09-6209a8dd9976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:17.270995 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-d83b54c5-0946-44ff-af09-6209a8dd9976 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:57:17.272167 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-d83b54c5-0946-44ff-af09-6209a8dd9976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:57:17.333591 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:17.338528 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 570/1112] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:57:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:57:17.361061 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86fd73-f4ef-456b-8871-dee261b1ae45 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:17.453718 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:17.461255 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86fd73-f4ef-456b-8871-dee261b1ae45 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:17.494324 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825583bf-4571-480b-8470-e9e46dd604b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:17.494946 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825583bf-4571-480b-8470-e9e46dd604b0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:17.495274 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825583bf-4571-480b-8470-e9e46dd604b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:17.495495 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825583bf-4571-480b-8470-e9e46dd604b0 None admin] PATH_INFO: `/v3/roles/47e170213d8049a28f8b8a0d736fdf6b` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:17.496932 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825583bf-4571-480b-8470-e9e46dd604b0 None admin] RBAC: Authorizing `identity:get_role(role_id=47e170213d8049a28f8b8a0d736fdf6b)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:17.498327 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825583bf-4571-480b-8470-e9e46dd604b0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:17.499633 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 543/1113] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:57:17 2022] GET /identity/v3/roles/47e170213d8049a28f8b8a0d736fdf6b => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:17.506286 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b54c5-0946-44ff-af09-6209a8dd9976 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:17.602443 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b54c5-0946-44ff-af09-6209a8dd9976 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:17.635682 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a0c891-d4eb-4812-823a-cbe79ea45390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:17.636315 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0c891-d4eb-4812-823a-cbe79ea45390 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:17.636629 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0c891-d4eb-4812-823a-cbe79ea45390 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:17.636909 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0c891-d4eb-4812-823a-cbe79ea45390 None admin] PATH_INFO: `/v3/users/410c52a476a545bbbb3b41293a06ece1` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:17.649675 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a0c891-d4eb-4812-823a-cbe79ea45390 None admin] RBAC: Authorizing `identity:get_user(user_id=410c52a476a545bbbb3b41293a06ece1)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:17.651108 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a0c891-d4eb-4812-823a-cbe79ea45390 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:17.652730 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 571/1114] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:57:17 2022] GET /identity/v3/users/410c52a476a545bbbb3b41293a06ece1 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:17.659128 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825583bf-4571-480b-8470-e9e46dd604b0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:17.760929 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825583bf-4571-480b-8470-e9e46dd604b0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:17.793496 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9da74-ff67-44b3-b99e-a01576245629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:17.794104 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9da74-ff67-44b3-b99e-a01576245629 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:17.794388 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9da74-ff67-44b3-b99e-a01576245629 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:17.794668 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9da74-ff67-44b3-b99e-a01576245629 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:17.824055 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:17.860922 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb9da74-ff67-44b3-b99e-a01576245629 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:17.862954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb9da74-ff67-44b3-b99e-a01576245629 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:17.956892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 544/1115] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:57:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 13:57:17.962683 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a0c891-d4eb-4812-823a-cbe79ea45390 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:18.019428 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:18.059084 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a0c891-d4eb-4812-823a-cbe79ea45390 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:18.091816 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85534b81-0cb2-4ab7-b4c1-7dea6d8b1385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:18.092450 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85534b81-0cb2-4ab7-b4c1-7dea6d8b1385 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:18.092767 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85534b81-0cb2-4ab7-b4c1-7dea6d8b1385 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:18.093046 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85534b81-0cb2-4ab7-b4c1-7dea6d8b1385 None admin] PATH_INFO: `/v3/projects/0f7ee50e24ca44fb9ebded4e154bb2f1` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:18.094186 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85534b81-0cb2-4ab7-b4c1-7dea6d8b1385 None admin] RBAC: Authorizing `identity:get_project(project_id=0f7ee50e24ca44fb9ebded4e154bb2f1)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:18.095600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85534b81-0cb2-4ab7-b4c1-7dea6d8b1385 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:18.097274 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 572/1116] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:57:17 2022] GET /identity/v3/projects/0f7ee50e24ca44fb9ebded4e154bb2f1 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:18.103220 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9da74-ff67-44b3-b99e-a01576245629 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:18.162629 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:18.200902 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9da74-ff67-44b3-b99e-a01576245629 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:18.232925 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38806cfa-da41-4210-aaf2-97cbb420b1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:18.233528 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38806cfa-da41-4210-aaf2-97cbb420b1df None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:18.233814 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38806cfa-da41-4210-aaf2-97cbb420b1df None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:18.234092 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38806cfa-da41-4210-aaf2-97cbb420b1df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:18.234875 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38806cfa-da41-4210-aaf2-97cbb420b1df None admin] RBAC: Adding query filter params (role.id=47e170213d8049a28f8b8a0d736fdf6b, scope.project.id=0f7ee50e24ca44fb9ebded4e154bb2f1, user.id=410c52a476a545bbbb3b41293a06ece1) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:18.235189 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38806cfa-da41-4210-aaf2-97cbb420b1df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:18.237119 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38806cfa-da41-4210-aaf2-97cbb420b1df None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:18.242350 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 545/1117] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:57:18 2022] GET /identity/v3/role_assignments?user.id=410c52a476a545bbbb3b41293a06ece1&scope.project.id=0f7ee50e24ca44fb9ebded4e154bb2f1&role.id=47e170213d8049a28f8b8a0d736fdf6b => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:20.190289 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc99ee9d-f924-4d33-8252-0aa930bca84e None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:20.190861 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc99ee9d-f924-4d33-8252-0aa930bca84e None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:20.190861 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc99ee9d-f924-4d33-8252-0aa930bca84e None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:20.192194 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 573/1118] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:57:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:57:20.197477 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32526316-01da-4b9f-9828-f949f8da2b2a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:20.197872 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32526316-01da-4b9f-9828-f949f8da2b2a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:20.198029 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32526316-01da-4b9f-9828-f949f8da2b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:20.215361 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-32526316-01da-4b9f-9828-f949f8da2b2a None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:57:20.216584 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-32526316-01da-4b9f-9828-f949f8da2b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:57:20.288976 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 546/1119] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:57:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:57:20.311686 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc99ee9d-f924-4d33-8252-0aa930bca84e None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:20.410248 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc99ee9d-f924-4d33-8252-0aa930bca84e None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:20.435134 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:20.446834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd06f5e8-2a40-45e5-8832-52d70bdcff5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:20.447412 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06f5e8-2a40-45e5-8832-52d70bdcff5d None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:20.447695 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06f5e8-2a40-45e5-8832-52d70bdcff5d None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:20.447971 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06f5e8-2a40-45e5-8832-52d70bdcff5d None admin] PATH_INFO: `/v3/roles/47e170213d8049a28f8b8a0d736fdf6b` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:20.449422 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd06f5e8-2a40-45e5-8832-52d70bdcff5d None admin] RBAC: Authorizing `identity:get_role(role_id=47e170213d8049a28f8b8a0d736fdf6b)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:20.450792 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd06f5e8-2a40-45e5-8832-52d70bdcff5d None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:20.452118 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 574/1120] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:57:20 2022] GET /identity/v3/roles/47e170213d8049a28f8b8a0d736fdf6b => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:20.457931 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32526316-01da-4b9f-9828-f949f8da2b2a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:20.512765 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:20.555702 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32526316-01da-4b9f-9828-f949f8da2b2a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:20.588924 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b424441-9479-4e41-b569-2b49f6b4fbef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:20.589584 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b424441-9479-4e41-b569-2b49f6b4fbef None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:20.589867 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b424441-9479-4e41-b569-2b49f6b4fbef None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:20.590145 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b424441-9479-4e41-b569-2b49f6b4fbef None admin] PATH_INFO: `/v3/users/410c52a476a545bbbb3b41293a06ece1` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:20.591297 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b424441-9479-4e41-b569-2b49f6b4fbef None admin] RBAC: Authorizing `identity:get_user(user_id=410c52a476a545bbbb3b41293a06ece1)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:20.592749 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b424441-9479-4e41-b569-2b49f6b4fbef None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:20.594346 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 547/1121] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:57:20 2022] GET /identity/v3/users/410c52a476a545bbbb3b41293a06ece1 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:20.600142 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd06f5e8-2a40-45e5-8832-52d70bdcff5d None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:20.697294 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd06f5e8-2a40-45e5-8832-52d70bdcff5d None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:20.730134 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659879e-63b6-4138-b8b8-9677329d16b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:20.730772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659879e-63b6-4138-b8b8-9677329d16b5 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:20.731057 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659879e-63b6-4138-b8b8-9677329d16b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:20.731338 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659879e-63b6-4138-b8b8-9677329d16b5 None admin] PATH_INFO: `/v3/projects/0f7ee50e24ca44fb9ebded4e154bb2f1` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:20.732521 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0659879e-63b6-4138-b8b8-9677329d16b5 None admin] RBAC: Authorizing `identity:get_project(project_id=0f7ee50e24ca44fb9ebded4e154bb2f1)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:20.733942 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0659879e-63b6-4138-b8b8-9677329d16b5 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:20.735587 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 575/1122] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:57:20 2022] GET /identity/v3/projects/0f7ee50e24ca44fb9ebded4e154bb2f1 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:20.741512 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b424441-9479-4e41-b569-2b49f6b4fbef None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:20.843364 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b424441-9479-4e41-b569-2b49f6b4fbef None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:20.865792 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:20.876368 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aaa67d-6e18-47be-9600-f5f9ebf78cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:20.877547 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aaa67d-6e18-47be-9600-f5f9ebf78cbc None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:20.877843 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aaa67d-6e18-47be-9600-f5f9ebf78cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:20.878386 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aaa67d-6e18-47be-9600-f5f9ebf78cbc None admin] PATH_INFO: `/v3/projects/0f7ee50e24ca44fb9ebded4e154bb2f1/users/410c52a476a545bbbb3b41293a06ece1/roles/47e170213d8049a28f8b8a0d736fdf6b` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:20.880584 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aaa67d-6e18-47be-9600-f5f9ebf78cbc None admin] RBAC: Authorizing `identity:create_grant(project_id=0f7ee50e24ca44fb9ebded4e154bb2f1, user_id=410c52a476a545bbbb3b41293a06ece1, role_id=47e170213d8049a28f8b8a0d736fdf6b)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:20.882013 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aaa67d-6e18-47be-9600-f5f9ebf78cbc None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:20.894498 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 548/1123] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 13:57:20 2022] PUT /identity/v3/projects/0f7ee50e24ca44fb9ebded4e154bb2f1/users/410c52a476a545bbbb3b41293a06ece1/roles/47e170213d8049a28f8b8a0d736fdf6b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:57:22.841732 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafb20e-48a4-446c-bd83-a15de6327f69 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:22.842155 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafb20e-48a4-446c-bd83-a15de6327f69 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:22.842295 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafb20e-48a4-446c-bd83-a15de6327f69 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:22.843658 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 576/1124] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:57:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:57:22.849332 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbeacae-7142-4ed8-a55c-46c94814bcb1 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:22.849746 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbeacae-7142-4ed8-a55c-46c94814bcb1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:22.849898 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbeacae-7142-4ed8-a55c-46c94814bcb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:22.868037 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-bcbeacae-7142-4ed8-a55c-46c94814bcb1 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:57:22.869272 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-bcbeacae-7142-4ed8-a55c-46c94814bcb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:57:22.941514 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 549/1125] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:57:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:57:22.964579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafb20e-48a4-446c-bd83-a15de6327f69 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:23.025266 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:23.064037 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafb20e-48a4-446c-bd83-a15de6327f69 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:23.096753 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f1747-b472-40ad-a9f7-d88a2c236af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:23.097321 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f1747-b472-40ad-a9f7-d88a2c236af9 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:23.097599 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f1747-b472-40ad-a9f7-d88a2c236af9 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:23.097872 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f1747-b472-40ad-a9f7-d88a2c236af9 None admin] PATH_INFO: `/v3/roles/47e170213d8049a28f8b8a0d736fdf6b` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:23.099292 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967f1747-b472-40ad-a9f7-d88a2c236af9 None admin] RBAC: Authorizing `identity:get_role(role_id=47e170213d8049a28f8b8a0d736fdf6b)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:23.100669 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967f1747-b472-40ad-a9f7-d88a2c236af9 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:23.101987 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 577/1126] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:57:22 2022] GET /identity/v3/roles/47e170213d8049a28f8b8a0d736fdf6b => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:23.107841 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbeacae-7142-4ed8-a55c-46c94814bcb1 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:23.212096 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbeacae-7142-4ed8-a55c-46c94814bcb1 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:23.226579 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:23.247125 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395df86b-3d50-433e-af2f-4f2299993a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:23.247772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395df86b-3d50-433e-af2f-4f2299993a49 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:23.248059 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395df86b-3d50-433e-af2f-4f2299993a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:23.248337 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395df86b-3d50-433e-af2f-4f2299993a49 None admin] PATH_INFO: `/v3/users/410c52a476a545bbbb3b41293a06ece1` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:23.249524 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395df86b-3d50-433e-af2f-4f2299993a49 None admin] RBAC: Authorizing `identity:get_user(user_id=410c52a476a545bbbb3b41293a06ece1)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:23.250944 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395df86b-3d50-433e-af2f-4f2299993a49 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:23.252561 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 550/1127] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:57:23 2022] GET /identity/v3/users/410c52a476a545bbbb3b41293a06ece1 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:23.258554 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f1747-b472-40ad-a9f7-d88a2c236af9 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:23.358889 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f1747-b472-40ad-a9f7-d88a2c236af9 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:23.386850 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:23.393528 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5947ec9-8045-4dd2-8012-7b2030055bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:23.394143 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5947ec9-8045-4dd2-8012-7b2030055bb0 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:23.394425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5947ec9-8045-4dd2-8012-7b2030055bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:23.394700 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5947ec9-8045-4dd2-8012-7b2030055bb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:23.459436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5947ec9-8045-4dd2-8012-7b2030055bb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:23.460903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5947ec9-8045-4dd2-8012-7b2030055bb0 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:23.556996 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 578/1128] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:57:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 13:57:23.563031 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395df86b-3d50-433e-af2f-4f2299993a49 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:23.660340 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395df86b-3d50-433e-af2f-4f2299993a49 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:23.694468 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0fd66c-1804-46e0-a9d2-ba8fe80d0654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:23.695105 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0fd66c-1804-46e0-a9d2-ba8fe80d0654 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:23.695390 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0fd66c-1804-46e0-a9d2-ba8fe80d0654 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:23.695670 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0fd66c-1804-46e0-a9d2-ba8fe80d0654 None admin] PATH_INFO: `/v3/projects/0f7ee50e24ca44fb9ebded4e154bb2f1` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:23.696847 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0fd66c-1804-46e0-a9d2-ba8fe80d0654 None admin] RBAC: Authorizing `identity:get_project(project_id=0f7ee50e24ca44fb9ebded4e154bb2f1)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:23.698273 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0fd66c-1804-46e0-a9d2-ba8fe80d0654 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:23.699931 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 551/1129] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:57:23 2022] GET /identity/v3/projects/0f7ee50e24ca44fb9ebded4e154bb2f1 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:23.706835 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5947ec9-8045-4dd2-8012-7b2030055bb0 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:23.735690 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:23.803924 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5947ec9-8045-4dd2-8012-7b2030055bb0 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:23.838140 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f5ef0-526b-4c8f-9699-d6bfb67032ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:23.838760 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f5ef0-526b-4c8f-9699-d6bfb67032ad None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:23.839044 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f5ef0-526b-4c8f-9699-d6bfb67032ad None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:23.839320 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f5ef0-526b-4c8f-9699-d6bfb67032ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:23.840109 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f5ef0-526b-4c8f-9699-d6bfb67032ad None admin] RBAC: Adding query filter params (role.id=47e170213d8049a28f8b8a0d736fdf6b, scope.project.id=0f7ee50e24ca44fb9ebded4e154bb2f1, user.id=410c52a476a545bbbb3b41293a06ece1) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:23.840423 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f5ef0-526b-4c8f-9699-d6bfb67032ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:23.841876 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f5ef0-526b-4c8f-9699-d6bfb67032ad None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:23.847555 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 579/1130] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:57:23 2022] GET /identity/v3/role_assignments?user.id=410c52a476a545bbbb3b41293a06ece1&scope.project.id=0f7ee50e24ca44fb9ebded4e154bb2f1&role.id=47e170213d8049a28f8b8a0d736fdf6b => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:25.832140 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eae32d-8e13-42b6-9576-23136d47cb45 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:25.832571 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eae32d-8e13-42b6-9576-23136d47cb45 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:25.832720 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eae32d-8e13-42b6-9576-23136d47cb45 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:25.834244 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 552/1131] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:57:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:57:25.839422 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ce5e5-279b-4293-bf13-eea0d8b08ed1 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:25.839840 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ce5e5-279b-4293-bf13-eea0d8b08ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:25.839981 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ce5e5-279b-4293-bf13-eea0d8b08ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:25.857682 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-ab5ce5e5-279b-4293-bf13-eea0d8b08ed1 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:57:25.858857 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-ab5ce5e5-279b-4293-bf13-eea0d8b08ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:57:25.922746 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 580/1132] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:57:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:57:25.945449 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eae32d-8e13-42b6-9576-23136d47cb45 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:25.993902 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:26.042459 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eae32d-8e13-42b6-9576-23136d47cb45 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:26.075260 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231073e0-09bd-407b-a8d3-289892237ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:26.075827 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231073e0-09bd-407b-a8d3-289892237ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:26.076106 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231073e0-09bd-407b-a8d3-289892237ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:26.076379 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231073e0-09bd-407b-a8d3-289892237ae6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:26.077542 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231073e0-09bd-407b-a8d3-289892237ae6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:26.078949 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231073e0-09bd-407b-a8d3-289892237ae6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:26.081166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 553/1133] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:57:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:26.087056 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ce5e5-279b-4293-bf13-eea0d8b08ed1 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:26.110796 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:26.185043 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ce5e5-279b-4293-bf13-eea0d8b08ed1 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:26.217918 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a604a4-4550-44fa-abfb-687b8934abab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:26.218541 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a604a4-4550-44fa-abfb-687b8934abab None admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:26.218825 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a604a4-4550-44fa-abfb-687b8934abab None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:26.219104 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a604a4-4550-44fa-abfb-687b8934abab None admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:26.219776 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a604a4-4550-44fa-abfb-687b8934abab None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:26.221256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a604a4-4550-44fa-abfb-687b8934abab None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:26.247549 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 581/1134] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 13:57:26 2022] POST /identity/v3/projects => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:57:28.237824 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d97760-10cb-4ae3-8e61-cd3e4846d91b None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:28.238376 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d97760-10cb-4ae3-8e61-cd3e4846d91b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:28.238376 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d97760-10cb-4ae3-8e61-cd3e4846d91b None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:28.239758 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 554/1135] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:57:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:57:28.244863 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a78bb-8b5a-4858-b6a6-4952f6d05efb None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:28.245277 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a78bb-8b5a-4858-b6a6-4952f6d05efb None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:28.245419 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a78bb-8b5a-4858-b6a6-4952f6d05efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:28.263817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-499a78bb-8b5a-4858-b6a6-4952f6d05efb None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:57:28.265050 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-499a78bb-8b5a-4858-b6a6-4952f6d05efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:57:28.330648 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 582/1136] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:57:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:57:28.354545 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d97760-10cb-4ae3-8e61-cd3e4846d91b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:28.454126 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d97760-10cb-4ae3-8e61-cd3e4846d91b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:28.470721 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:28.488521 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9164244-3bf0-499d-9e60-9f90e8de33f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:28.489111 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9164244-3bf0-499d-9e60-9f90e8de33f2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:28.489398 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9164244-3bf0-499d-9e60-9f90e8de33f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:28.489677 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9164244-3bf0-499d-9e60-9f90e8de33f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:28.490828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9164244-3bf0-499d-9e60-9f90e8de33f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:28.492263 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9164244-3bf0-499d-9e60-9f90e8de33f2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:28.493863 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 555/1137] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:57:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:28.499504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a78bb-8b5a-4858-b6a6-4952f6d05efb None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:28.508618 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:28.600241 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a78bb-8b5a-4858-b6a6-4952f6d05efb None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:28.633412 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29c4517-ea16-49bb-ba0d-db37a0a8c603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:28.634032 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c4517-ea16-49bb-ba0d-db37a0a8c603 None admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:28.634310 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c4517-ea16-49bb-ba0d-db37a0a8c603 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:28.634583 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c4517-ea16-49bb-ba0d-db37a0a8c603 None admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:28.635252 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29c4517-ea16-49bb-ba0d-db37a0a8c603 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:28.636680 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29c4517-ea16-49bb-ba0d-db37a0a8c603 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:28.657305 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 583/1138] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 13:57:28 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:57:30.703795 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307a321-ff2c-4904-bd0c-3752ab029d3a None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:30.704825 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307a321-ff2c-4904-bd0c-3752ab029d3a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:30.704946 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307a321-ff2c-4904-bd0c-3752ab029d3a None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:30.706260 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 556/1139] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:57:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:57:30.711515 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d17a08-4a9f-4aa9-8f7e-759d9b1fc984 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:30.711947 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d17a08-4a9f-4aa9-8f7e-759d9b1fc984 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:30.712078 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d17a08-4a9f-4aa9-8f7e-759d9b1fc984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:30.730349 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-f4d17a08-4a9f-4aa9-8f7e-759d9b1fc984 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:57:30.731534 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-f4d17a08-4a9f-4aa9-8f7e-759d9b1fc984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:57:30.786679 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:30.797573 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 584/1140] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:57:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:57:30.820455 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9307a321-ff2c-4904-bd0c-3752ab029d3a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:30.920671 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9307a321-ff2c-4904-bd0c-3752ab029d3a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:30.952606 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e139c5-3fd6-40e2-8d3b-c40057b8229e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:30.953237 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e139c5-3fd6-40e2-8d3b-c40057b8229e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:30.953521 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e139c5-3fd6-40e2-8d3b-c40057b8229e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:30.953851 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e139c5-3fd6-40e2-8d3b-c40057b8229e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:30.964128 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e139c5-3fd6-40e2-8d3b-c40057b8229e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:57:30.969363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-82e139c5-3fd6-40e2-8d3b-c40057b8229e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:57:30.969363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:30.969363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:30.969363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:30.969363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:30.969363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:30.969363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:30.969363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:30.969363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:30.969363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:30.969363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:30.969363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:30.969363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:57:30.969363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:57:30.969363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:57:30.969363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:57:30.971112 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:57:30.971112 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:57:30.971112 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:57:30.971112 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:57:30.971112 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:30.971112 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:30.971112 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:30.971112 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:30.971112 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:30.971112 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:30.971112 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:30.971112 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:30.971112 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:30.971112 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:30.971112 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:30.971112 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:30.972084 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:30.972084 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:30.972084 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:30.972084 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:30.972084 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:57:30.972084 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:57:30.972084 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:57:30.972084 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:57:30.972084 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:57:30.972084 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:57:30.972084 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:57:30.972084 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:57:30.972084 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 557/1141] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 13:57:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:30.977269 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d17a08-4a9f-4aa9-8f7e-759d9b1fc984 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:31.072709 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d17a08-4a9f-4aa9-8f7e-759d9b1fc984 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:31.105485 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e9ba48-05ef-4f78-9b54-bf46433b5a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:31.106095 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9ba48-05ef-4f78-9b54-bf46433b5a60 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:31.106376 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9ba48-05ef-4f78-9b54-bf46433b5a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:31.106650 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9ba48-05ef-4f78-9b54-bf46433b5a60 None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:31.107373 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e9ba48-05ef-4f78-9b54-bf46433b5a60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:31.107680 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e9ba48-05ef-4f78-9b54-bf46433b5a60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:31.109076 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e9ba48-05ef-4f78-9b54-bf46433b5a60 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:31.116934 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 585/1142] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:57:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:57:31.122609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e139c5-3fd6-40e2-8d3b-c40057b8229e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:31.169470 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:31.219600 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e139c5-3fd6-40e2-8d3b-c40057b8229e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:31.252211 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e70706-041b-40f9-89f4-ef0ffd2c084a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:31.253121 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e70706-041b-40f9-89f4-ef0ffd2c084a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:31.253398 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e70706-041b-40f9-89f4-ef0ffd2c084a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:31.253668 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e70706-041b-40f9-89f4-ef0ffd2c084a None admin] PATH_INFO: `/v3/users/76452e2856dc47eb9f8c48f864d88f17` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:31.265986 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e70706-041b-40f9-89f4-ef0ffd2c084a None admin] RBAC: Authorizing `identity:get_user(user_id=76452e2856dc47eb9f8c48f864d88f17)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:31.267442 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e70706-041b-40f9-89f4-ef0ffd2c084a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:31.269047 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 558/1143] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:57:31 2022] GET /identity/v3/users/76452e2856dc47eb9f8c48f864d88f17 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:31.274846 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e9ba48-05ef-4f78-9b54-bf46433b5a60 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:31.328659 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:31.375296 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e9ba48-05ef-4f78-9b54-bf46433b5a60 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:31.408322 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a26fb83-27a8-44d7-82a2-0cbab5f08846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:31.408962 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26fb83-27a8-44d7-82a2-0cbab5f08846 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:31.409245 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26fb83-27a8-44d7-82a2-0cbab5f08846 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:31.409522 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26fb83-27a8-44d7-82a2-0cbab5f08846 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:31.473943 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a26fb83-27a8-44d7-82a2-0cbab5f08846 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:31.475312 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a26fb83-27a8-44d7-82a2-0cbab5f08846 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:31.526941 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:31.570943 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 586/1144] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:57:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 13:57:31.576884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e70706-041b-40f9-89f4-ef0ffd2c084a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:31.671321 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e70706-041b-40f9-89f4-ef0ffd2c084a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:31.674107 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:31.704503 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc81372-e903-40c7-8206-4332395aae73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:31.705144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc81372-e903-40c7-8206-4332395aae73 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:31.705423 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc81372-e903-40c7-8206-4332395aae73 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:31.705694 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc81372-e903-40c7-8206-4332395aae73 None admin] PATH_INFO: `/v3/projects/a29ae1548e9b481db0405d6c5ababea8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:31.706965 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc81372-e903-40c7-8206-4332395aae73 None admin] RBAC: Authorizing `identity:get_project(project_id=a29ae1548e9b481db0405d6c5ababea8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:31.708384 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc81372-e903-40c7-8206-4332395aae73 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:31.710037 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 559/1145] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:57:31 2022] GET /identity/v3/projects/a29ae1548e9b481db0405d6c5ababea8 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:31.716241 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a26fb83-27a8-44d7-82a2-0cbab5f08846 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:31.814028 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a26fb83-27a8-44d7-82a2-0cbab5f08846 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:31.849969 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d75c8-ff47-40ad-b647-5a41b8759fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:31.850586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d75c8-ff47-40ad-b647-5a41b8759fad None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:31.850870 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d75c8-ff47-40ad-b647-5a41b8759fad None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:31.851146 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d75c8-ff47-40ad-b647-5a41b8759fad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:31.851938 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d75c8-ff47-40ad-b647-5a41b8759fad None admin] RBAC: Adding query filter params (role.id=58f91e3a13664dabb18c6227a1658fa6, scope.project.id=a29ae1548e9b481db0405d6c5ababea8, user.id=76452e2856dc47eb9f8c48f864d88f17) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:31.852247 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d75c8-ff47-40ad-b647-5a41b8759fad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:31.853729 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d75c8-ff47-40ad-b647-5a41b8759fad None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:31.859046 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 587/1146] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:57:31 2022] GET /identity/v3/role_assignments?user.id=76452e2856dc47eb9f8c48f864d88f17&scope.project.id=a29ae1548e9b481db0405d6c5ababea8&role.id=58f91e3a13664dabb18c6227a1658fa6 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:33.815316 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a43e9d-58ad-4c42-bfba-e3ba45486eb4 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:33.815864 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a43e9d-58ad-4c42-bfba-e3ba45486eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:33.815969 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a43e9d-58ad-4c42-bfba-e3ba45486eb4 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:33.817447 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 560/1147] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:57:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:57:33.823652 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736ee1e-cf23-43d4-8025-4f116048cac6 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:33.824054 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736ee1e-cf23-43d4-8025-4f116048cac6 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:33.824211 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736ee1e-cf23-43d4-8025-4f116048cac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:33.841760 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-3736ee1e-cf23-43d4-8025-4f116048cac6 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:57:33.842927 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-3736ee1e-cf23-43d4-8025-4f116048cac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:57:33.855756 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:33.907853 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 588/1148] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:57:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:57:33.930086 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a43e9d-58ad-4c42-bfba-e3ba45486eb4 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:34.028065 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a43e9d-58ad-4c42-bfba-e3ba45486eb4 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:34.061320 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4890f65-c96a-4678-bfcb-617cb9670093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:34.061946 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4890f65-c96a-4678-bfcb-617cb9670093 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:34.062231 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4890f65-c96a-4678-bfcb-617cb9670093 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:34.062504 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4890f65-c96a-4678-bfcb-617cb9670093 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:34.072472 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4890f65-c96a-4678-bfcb-617cb9670093 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:57:34.077250 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-a4890f65-c96a-4678-bfcb-617cb9670093 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:57:34.077250 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:34.077250 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:34.077250 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:34.077250 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:34.077250 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:34.077250 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:34.077250 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:34.077250 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:34.077250 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:34.077250 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:34.077250 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:34.077250 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:57:34.077250 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:57:34.077250 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:57:34.077250 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:57:34.078322 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:57:34.078322 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:57:34.078322 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:57:34.078322 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:57:34.078322 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:34.078322 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:34.078322 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:34.078322 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:34.078322 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:34.078322 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:34.078322 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:34.078322 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:34.078322 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:34.078322 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:34.078322 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:34.078322 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:34.079270 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:34.079270 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:34.079270 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:34.079270 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:34.079270 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:57:34.079270 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:57:34.079270 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:57:34.079270 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:57:34.079270 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:57:34.079270 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:57:34.079270 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:57:34.079270 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:57:34.079270 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 561/1149] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 13:57:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:34.084246 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736ee1e-cf23-43d4-8025-4f116048cac6 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:34.180415 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736ee1e-cf23-43d4-8025-4f116048cac6 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:34.212628 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bccd43-5839-4715-9aae-09c5777e403c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:34.213236 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bccd43-5839-4715-9aae-09c5777e403c None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:34.213522 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bccd43-5839-4715-9aae-09c5777e403c None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:34.213800 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bccd43-5839-4715-9aae-09c5777e403c None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:34.214519 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bccd43-5839-4715-9aae-09c5777e403c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:34.214829 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bccd43-5839-4715-9aae-09c5777e403c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:34.216180 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bccd43-5839-4715-9aae-09c5777e403c None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:34.224462 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 589/1150] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:57:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:34.230324 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4890f65-c96a-4678-bfcb-617cb9670093 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:34.261377 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:34.327732 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4890f65-c96a-4678-bfcb-617cb9670093 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:34.361327 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98565b9-2301-4416-940f-06f7b2ac356c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:34.361950 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98565b9-2301-4416-940f-06f7b2ac356c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:34.362233 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98565b9-2301-4416-940f-06f7b2ac356c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:34.362514 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98565b9-2301-4416-940f-06f7b2ac356c None admin] PATH_INFO: `/v3/users/76452e2856dc47eb9f8c48f864d88f17` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:34.363653 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98565b9-2301-4416-940f-06f7b2ac356c None admin] RBAC: Authorizing `identity:get_user(user_id=76452e2856dc47eb9f8c48f864d88f17)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:34.365102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98565b9-2301-4416-940f-06f7b2ac356c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:34.366670 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 562/1151] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:57:34 2022] GET /identity/v3/users/76452e2856dc47eb9f8c48f864d88f17 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:34.372492 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bccd43-5839-4715-9aae-09c5777e403c None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:34.385605 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:34.471900 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bccd43-5839-4715-9aae-09c5777e403c None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:34.506688 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70f6a9d-e5e9-4b42-80c5-089bd3775c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:34.507321 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f6a9d-e5e9-4b42-80c5-089bd3775c71 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:34.507602 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f6a9d-e5e9-4b42-80c5-089bd3775c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:34.507877 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f6a9d-e5e9-4b42-80c5-089bd3775c71 None admin] PATH_INFO: `/v3/projects/a29ae1548e9b481db0405d6c5ababea8` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:34.509032 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70f6a9d-e5e9-4b42-80c5-089bd3775c71 None admin] RBAC: Authorizing `identity:get_project(project_id=a29ae1548e9b481db0405d6c5ababea8)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:34.510444 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70f6a9d-e5e9-4b42-80c5-089bd3775c71 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:34.512095 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 590/1152] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:57:34 2022] GET /identity/v3/projects/a29ae1548e9b481db0405d6c5ababea8 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:34.519239 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98565b9-2301-4416-940f-06f7b2ac356c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:34.613990 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:34.617001 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98565b9-2301-4416-940f-06f7b2ac356c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:34.650343 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be082c-cc1c-48cc-8eba-54f7f22e4860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:34.650997 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be082c-cc1c-48cc-8eba-54f7f22e4860 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:34.651277 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be082c-cc1c-48cc-8eba-54f7f22e4860 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:34.651550 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be082c-cc1c-48cc-8eba-54f7f22e4860 None admin] PATH_INFO: `/v3/projects/a29ae1548e9b481db0405d6c5ababea8/users/76452e2856dc47eb9f8c48f864d88f17/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:34.653477 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be082c-cc1c-48cc-8eba-54f7f22e4860 None admin] RBAC: Authorizing `identity:create_grant(project_id=a29ae1548e9b481db0405d6c5ababea8, user_id=76452e2856dc47eb9f8c48f864d88f17, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:34.654894 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be082c-cc1c-48cc-8eba-54f7f22e4860 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:34.668117 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 563/1153] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 13:57:34 2022] PUT /identity/v3/projects/a29ae1548e9b481db0405d6c5ababea8/users/76452e2856dc47eb9f8c48f864d88f17/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:57:36.634279 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d2ab4-e1fe-4984-90c3-fb46e17098a7 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:36.634825 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d2ab4-e1fe-4984-90c3-fb46e17098a7 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:36.634825 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d2ab4-e1fe-4984-90c3-fb46e17098a7 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:36.636173 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 591/1154] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:57:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:57:36.641390 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a66b34-c058-4ed0-9dc3-73bfc40cb2cc None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:36.642284 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a66b34-c058-4ed0-9dc3-73bfc40cb2cc None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:36.642564 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a66b34-c058-4ed0-9dc3-73bfc40cb2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:36.660120 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-73a66b34-c058-4ed0-9dc3-73bfc40cb2cc None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:57:36.661332 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-73a66b34-c058-4ed0-9dc3-73bfc40cb2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:57:36.725038 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 564/1155] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:57:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:57:36.748158 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d2ab4-e1fe-4984-90c3-fb46e17098a7 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:36.826263 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:36.848424 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d2ab4-e1fe-4984-90c3-fb46e17098a7 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:36.881273 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d428008d-f85e-43e3-9b32-f3afb066b740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:36.881905 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d428008d-f85e-43e3-9b32-f3afb066b740 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:36.882187 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d428008d-f85e-43e3-9b32-f3afb066b740 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:36.882464 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d428008d-f85e-43e3-9b32-f3afb066b740 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:36.892771 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d428008d-f85e-43e3-9b32-f3afb066b740 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:57:36.897562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-d428008d-f85e-43e3-9b32-f3afb066b740 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:57:36.897562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:36.897562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:36.897562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:36.897562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:36.897562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:36.897562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:36.897562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:36.897562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:36.897562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:36.897562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:36.897562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:36.897562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:57:36.897562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:57:36.897562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:57:36.897562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:57:36.898608 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:57:36.898608 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:57:36.898608 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:57:36.898608 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:57:36.898608 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:36.898608 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:36.898608 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:36.898608 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:36.898608 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:36.898608 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:36.898608 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:36.898608 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:36.898608 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:36.898608 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:36.898608 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:36.898608 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:36.899808 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:36.899808 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:36.899808 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:36.899808 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:36.899808 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:57:36.899808 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:57:36.899808 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:57:36.899808 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:57:36.899808 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:57:36.899808 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:57:36.899808 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:57:36.899808 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:57:36.899808 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 592/1156] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 13:57:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:36.904824 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a66b34-c058-4ed0-9dc3-73bfc40cb2cc None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:36.999197 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:37.010255 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a66b34-c058-4ed0-9dc3-73bfc40cb2cc None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:37.042731 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decc5862-7114-412e-ada2-19bb00eb9eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:37.043341 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc5862-7114-412e-ada2-19bb00eb9eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:37.043621 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc5862-7114-412e-ada2-19bb00eb9eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:37.043894 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc5862-7114-412e-ada2-19bb00eb9eb8 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:37.044627 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decc5862-7114-412e-ada2-19bb00eb9eb8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:37.044932 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decc5862-7114-412e-ada2-19bb00eb9eb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:37.046278 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decc5862-7114-412e-ada2-19bb00eb9eb8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:37.054640 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 565/1157] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:57:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:37.060449 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d428008d-f85e-43e3-9b32-f3afb066b740 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:37.157544 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d428008d-f85e-43e3-9b32-f3afb066b740 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:37.189935 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b1236-4c89-4100-9b04-e4d1abe10749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:37.190564 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b1236-4c89-4100-9b04-e4d1abe10749 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:37.190850 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b1236-4c89-4100-9b04-e4d1abe10749 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:37.191128 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b1236-4c89-4100-9b04-e4d1abe10749 None admin] PATH_INFO: `/v3/users/76452e2856dc47eb9f8c48f864d88f17` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:37.192272 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329b1236-4c89-4100-9b04-e4d1abe10749 None admin] RBAC: Authorizing `identity:get_user(user_id=76452e2856dc47eb9f8c48f864d88f17)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:37.193723 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329b1236-4c89-4100-9b04-e4d1abe10749 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:37.195291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 593/1158] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:57:37 2022] GET /identity/v3/users/76452e2856dc47eb9f8c48f864d88f17 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:37.200836 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decc5862-7114-412e-ada2-19bb00eb9eb8 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:37.296840 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decc5862-7114-412e-ada2-19bb00eb9eb8 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:37.331456 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8993c-2fa0-4240-9c9f-58ef9282b239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:37.332087 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8993c-2fa0-4240-9c9f-58ef9282b239 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:37.332371 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8993c-2fa0-4240-9c9f-58ef9282b239 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:37.332676 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8993c-2fa0-4240-9c9f-58ef9282b239 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:37.355581 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:37.401674 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a8993c-2fa0-4240-9c9f-58ef9282b239 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:37.403118 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a8993c-2fa0-4240-9c9f-58ef9282b239 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:37.498024 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 566/1159] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:57:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 13:57:37.504322 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b1236-4c89-4100-9b04-e4d1abe10749 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:37.514360 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:37.602949 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b1236-4c89-4100-9b04-e4d1abe10749 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:37.635239 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c2ac12-a5c5-460c-b67e-c9944d34a79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:37.635879 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c2ac12-a5c5-460c-b67e-c9944d34a79b None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:37.636161 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c2ac12-a5c5-460c-b67e-c9944d34a79b None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:37.636439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c2ac12-a5c5-460c-b67e-c9944d34a79b None admin] PATH_INFO: `/v3/projects/a29ae1548e9b481db0405d6c5ababea8` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:37.637610 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c2ac12-a5c5-460c-b67e-c9944d34a79b None admin] RBAC: Authorizing `identity:get_project(project_id=a29ae1548e9b481db0405d6c5ababea8)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:37.639068 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c2ac12-a5c5-460c-b67e-c9944d34a79b None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:37.640751 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 594/1160] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:57:37 2022] GET /identity/v3/projects/a29ae1548e9b481db0405d6c5ababea8 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:37.647511 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8993c-2fa0-4240-9c9f-58ef9282b239 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:37.698685 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:37.744339 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8993c-2fa0-4240-9c9f-58ef9282b239 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:37.776987 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e20ebf-b2a8-48e4-9abc-9178266893a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:37.777535 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e20ebf-b2a8-48e4-9abc-9178266893a4 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:37.777815 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e20ebf-b2a8-48e4-9abc-9178266893a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:37.778087 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e20ebf-b2a8-48e4-9abc-9178266893a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:37.778869 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e20ebf-b2a8-48e4-9abc-9178266893a4 None admin] RBAC: Adding query filter params (role.id=58f91e3a13664dabb18c6227a1658fa6, scope.project.id=a29ae1548e9b481db0405d6c5ababea8, user.id=76452e2856dc47eb9f8c48f864d88f17) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:37.779179 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e20ebf-b2a8-48e4-9abc-9178266893a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:37.780620 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e20ebf-b2a8-48e4-9abc-9178266893a4 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:37.786796 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 567/1161] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:57:37 2022] GET /identity/v3/role_assignments?user.id=76452e2856dc47eb9f8c48f864d88f17&scope.project.id=a29ae1548e9b481db0405d6c5ababea8&role.id=58f91e3a13664dabb18c6227a1658fa6 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:39.765143 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc33234-c861-41f9-8fdf-87f052c8a2cb None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:39.765701 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc33234-c861-41f9-8fdf-87f052c8a2cb None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:39.765701 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc33234-c861-41f9-8fdf-87f052c8a2cb None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:39.767025 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 595/1162] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:57:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:57:39.772038 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f28030-f8fc-4c10-a79f-aee576464aac None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:39.772436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f28030-f8fc-4c10-a79f-aee576464aac None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:39.772628 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f28030-f8fc-4c10-a79f-aee576464aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:39.790151 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-90f28030-f8fc-4c10-a79f-aee576464aac None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:57:39.791344 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-90f28030-f8fc-4c10-a79f-aee576464aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:57:39.856670 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 568/1163] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:57:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:57:39.879747 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc33234-c861-41f9-8fdf-87f052c8a2cb None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:39.959873 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:39.984545 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc33234-c861-41f9-8fdf-87f052c8a2cb None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:40.017399 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f117e50e-b215-4035-bb75-9c5eb5ffb412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:40.017946 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117e50e-b215-4035-bb75-9c5eb5ffb412 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:40.018225 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117e50e-b215-4035-bb75-9c5eb5ffb412 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:40.018497 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117e50e-b215-4035-bb75-9c5eb5ffb412 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:40.081844 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f117e50e-b215-4035-bb75-9c5eb5ffb412 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:40.083260 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f117e50e-b215-4035-bb75-9c5eb5ffb412 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:40.152623 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:40.178023 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 596/1164] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:57:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 13:57:40.184257 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f28030-f8fc-4c10-a79f-aee576464aac None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:40.233420 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:40.285041 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f28030-f8fc-4c10-a79f-aee576464aac None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:40.317657 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aeeba0-9ab3-449f-bfc7-2ca3ba19edde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:40.318296 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aeeba0-9ab3-449f-bfc7-2ca3ba19edde None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:40.318584 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aeeba0-9ab3-449f-bfc7-2ca3ba19edde None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:40.318864 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aeeba0-9ab3-449f-bfc7-2ca3ba19edde None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:40.325402 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aeeba0-9ab3-449f-bfc7-2ca3ba19edde None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:40.326833 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aeeba0-9ab3-449f-bfc7-2ca3ba19edde None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:40.338078 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-75aeeba0-9ab3-449f-bfc7-2ca3ba19edde None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 01 13:57:40.338078 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:40.338078 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 01 13:57:40.338078 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 01 13:57:40.338078 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:57:40.338078 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:57:40.338078 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:57:40.338078 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:57:40.338078 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 01 13:57:40.338078 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:57:40.338078 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:57:40.338078 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:57:40.338078 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:40.338078 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:40.338078 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:40.338078 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:40.338078 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:40.339233 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:40.339233 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:40.339233 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:40.339233 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:40.339233 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:40.339233 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:40.339233 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:57:40.339233 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:57:40.339233 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:57:40.339233 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:57:40.339233 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:40.339233 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:40.339233 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:40.339233 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:40.339233 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:40.339233 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:40.340083 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:40.340083 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:40.340083 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:40.340083 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:40.340083 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:40.340083 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:40.340083 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:40.340083 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:40.340083 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:40.340083 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:40.340083 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 01 13:57:40.340083 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:57:40.340083 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 01 13:57:40.340083 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:57:40.340083 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 569/1165] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 1 13:57:40 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:40.345753 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f117e50e-b215-4035-bb75-9c5eb5ffb412 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:40.442382 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f117e50e-b215-4035-bb75-9c5eb5ffb412 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:40.475797 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173217f2-d3d8-49f7-a1d0-c67f50eb4d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:40.476406 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173217f2-d3d8-49f7-a1d0-c67f50eb4d8f None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:40.476720 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173217f2-d3d8-49f7-a1d0-c67f50eb4d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:40.477002 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173217f2-d3d8-49f7-a1d0-c67f50eb4d8f None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:40.477694 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173217f2-d3d8-49f7-a1d0-c67f50eb4d8f None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:40.478005 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173217f2-d3d8-49f7-a1d0-c67f50eb4d8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:40.479367 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173217f2-d3d8-49f7-a1d0-c67f50eb4d8f None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:40.486347 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 597/1166] 10.222.0.48 () {56 vars in 1115 bytes} [Mon Aug 1 13:57:40 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:42.538658 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b780ca-0a90-40ec-97bc-a91ae1288255 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:42.539204 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b780ca-0a90-40ec-97bc-a91ae1288255 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:42.539204 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b780ca-0a90-40ec-97bc-a91ae1288255 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:42.540585 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 570/1167] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:57:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:57:42.545874 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3a547-e84b-4823-958d-6a5461dd5898 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:42.546268 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3a547-e84b-4823-958d-6a5461dd5898 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:42.546440 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3a547-e84b-4823-958d-6a5461dd5898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:42.564430 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-07e3a547-e84b-4823-958d-6a5461dd5898 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:57:42.565626 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-07e3a547-e84b-4823-958d-6a5461dd5898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:57:42.594727 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:42.630761 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 598/1168] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:57:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:57:42.652888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b780ca-0a90-40ec-97bc-a91ae1288255 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:42.757287 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b780ca-0a90-40ec-97bc-a91ae1288255 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:42.763844 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:42.790954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd008af-5361-42fb-b047-5d7d52d5eb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:42.791511 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd008af-5361-42fb-b047-5d7d52d5eb04 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:42.791798 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd008af-5361-42fb-b047-5d7d52d5eb04 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:42.792078 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd008af-5361-42fb-b047-5d7d52d5eb04 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:42.792824 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd008af-5361-42fb-b047-5d7d52d5eb04 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:42.794188 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd008af-5361-42fb-b047-5d7d52d5eb04 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:42.810213 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 571/1169] 10.222.0.48 () {60 vars in 1138 bytes} [Mon Aug 1 13:57:42 2022] POST /identity/v3/domains => generated 267 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:57:44.811577 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f6764-509c-4888-8aaf-2db9a9162dfc None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:44.812137 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f6764-509c-4888-8aaf-2db9a9162dfc None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:44.812137 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f6764-509c-4888-8aaf-2db9a9162dfc None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:44.814058 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 599/1170] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:57:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:57:44.819326 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a060a-7fd5-4ed1-9ffd-8522d8164a00 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:44.819750 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a060a-7fd5-4ed1-9ffd-8522d8164a00 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:44.819885 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a060a-7fd5-4ed1-9ffd-8522d8164a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:44.837984 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-012a060a-7fd5-4ed1-9ffd-8522d8164a00 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:57:44.839153 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-012a060a-7fd5-4ed1-9ffd-8522d8164a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:57:44.902695 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 572/1171] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:57:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:57:44.925437 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f6764-509c-4888-8aaf-2db9a9162dfc None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:45.023116 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f6764-509c-4888-8aaf-2db9a9162dfc None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:45.056472 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4300868-2240-44fe-aa0a-10610d535524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:45.057139 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4300868-2240-44fe-aa0a-10610d535524 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:45.057422 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4300868-2240-44fe-aa0a-10610d535524 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:45.057699 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4300868-2240-44fe-aa0a-10610d535524 None admin] PATH_INFO: `/v3/domains/ab5529d414eb450ebd3cd4e5ff22311c` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:45.068305 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4300868-2240-44fe-aa0a-10610d535524 None admin] RBAC: Authorizing `identity:get_domain(domain_id=ab5529d414eb450ebd3cd4e5ff22311c)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:45.069722 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4300868-2240-44fe-aa0a-10610d535524 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:45.071304 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 600/1172] 10.222.0.48 () {56 vars in 1183 bytes} [Mon Aug 1 13:57:44 2022] GET /identity/v3/domains/ab5529d414eb450ebd3cd4e5ff22311c => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:45.077161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a060a-7fd5-4ed1-9ffd-8522d8164a00 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:45.166898 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:45.173561 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a060a-7fd5-4ed1-9ffd-8522d8164a00 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:45.206798 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26508fa3-ced0-4aa7-bd45-9fce5206c08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:45.207412 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26508fa3-ced0-4aa7-bd45-9fce5206c08e None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:45.207697 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26508fa3-ced0-4aa7-bd45-9fce5206c08e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:45.207974 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26508fa3-ced0-4aa7-bd45-9fce5206c08e None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:45.208691 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26508fa3-ced0-4aa7-bd45-9fce5206c08e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:45.210123 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26508fa3-ced0-4aa7-bd45-9fce5206c08e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:45.235350 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 573/1173] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 13:57:45 2022] POST /identity/v3/projects => generated 361 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:57:47.264539 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037088e7-9d23-4905-9a64-242913e99eae None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:47.265064 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037088e7-9d23-4905-9a64-242913e99eae None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:47.265155 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037088e7-9d23-4905-9a64-242913e99eae None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:47.266996 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 601/1174] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:57:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:57:47.272197 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040856ec-6dcf-494e-9958-c57019f6d8bf None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:47.272647 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040856ec-6dcf-494e-9958-c57019f6d8bf None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:47.272794 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040856ec-6dcf-494e-9958-c57019f6d8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:47.291025 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-040856ec-6dcf-494e-9958-c57019f6d8bf None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:57:47.292209 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-040856ec-6dcf-494e-9958-c57019f6d8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:57:47.360674 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 574/1175] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:57:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:57:47.384991 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037088e7-9d23-4905-9a64-242913e99eae None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:47.413598 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:47.487039 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037088e7-9d23-4905-9a64-242913e99eae None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:47.519664 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b479198-da58-4547-afdb-c561744f0c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:47.520239 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b479198-da58-4547-afdb-c561744f0c9e None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:47.520544 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b479198-da58-4547-afdb-c561744f0c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:47.520825 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b479198-da58-4547-afdb-c561744f0c9e None admin] PATH_INFO: `/v3/domains/ab5529d414eb450ebd3cd4e5ff22311c` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:47.521971 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b479198-da58-4547-afdb-c561744f0c9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=ab5529d414eb450ebd3cd4e5ff22311c)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:47.523391 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b479198-da58-4547-afdb-c561744f0c9e None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:47.524992 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 602/1176] 10.222.0.48 () {56 vars in 1183 bytes} [Mon Aug 1 13:57:47 2022] GET /identity/v3/domains/ab5529d414eb450ebd3cd4e5ff22311c => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:47.530777 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040856ec-6dcf-494e-9958-c57019f6d8bf None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:47.606464 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:47.628405 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040856ec-6dcf-494e-9958-c57019f6d8bf None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:47.661869 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b595d7-14a0-4e32-82ce-08cde7a39757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:47.662483 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b595d7-14a0-4e32-82ce-08cde7a39757 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:47.662770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b595d7-14a0-4e32-82ce-08cde7a39757 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:47.663051 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b595d7-14a0-4e32-82ce-08cde7a39757 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:47.663742 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b595d7-14a0-4e32-82ce-08cde7a39757 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:47.665195 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b595d7-14a0-4e32-82ce-08cde7a39757 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:47.685310 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 575/1177] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 13:57:47 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:57:49.695167 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644daa5-5dd0-4784-9323-212cdcd64cc7 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:49.695800 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644daa5-5dd0-4784-9323-212cdcd64cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:49.695800 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644daa5-5dd0-4784-9323-212cdcd64cc7 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:49.697090 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 603/1178] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:57:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:57:49.702131 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5b86f-5de5-487c-89b7-fe4d239a9951 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:49.702569 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5b86f-5de5-487c-89b7-fe4d239a9951 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:49.702702 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5b86f-5de5-487c-89b7-fe4d239a9951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:49.720450 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b4b5b86f-5de5-487c-89b7-fe4d239a9951 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:57:49.721663 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b4b5b86f-5de5-487c-89b7-fe4d239a9951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:57:49.786507 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 576/1179] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:57:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:57:49.809460 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644daa5-5dd0-4784-9323-212cdcd64cc7 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:49.903303 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:49.910916 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644daa5-5dd0-4784-9323-212cdcd64cc7 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:49.943070 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93acda01-8b4a-4425-8e6b-282505df668e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:49.943705 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93acda01-8b4a-4425-8e6b-282505df668e None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:49.943987 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93acda01-8b4a-4425-8e6b-282505df668e None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:49.944261 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93acda01-8b4a-4425-8e6b-282505df668e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:49.954601 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93acda01-8b4a-4425-8e6b-282505df668e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:57:49.959382 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-93acda01-8b4a-4425-8e6b-282505df668e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:57:49.959382 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:49.959382 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:49.959382 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:49.959382 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:49.959382 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:49.959382 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:49.959382 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:49.959382 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:49.959382 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:49.959382 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:49.959382 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:49.959382 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:57:49.959382 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:57:49.959382 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:57:49.959382 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:57:49.960571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:57:49.960571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:57:49.960571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:57:49.960571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:57:49.960571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:49.960571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:49.960571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:49.960571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:49.960571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:49.960571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:49.960571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:49.960571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:49.960571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:49.960571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:49.960571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:49.960571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:49.961725 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:49.961725 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:49.961725 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:49.961725 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:49.961725 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:57:49.961725 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:57:49.961725 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:57:49.961725 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:57:49.961725 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:57:49.961725 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:57:49.961725 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:57:49.961725 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:57:49.961725 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 604/1180] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 13:57:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:49.966684 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b5b86f-5de5-487c-89b7-fe4d239a9951 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:50.032472 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:50.068934 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b5b86f-5de5-487c-89b7-fe4d239a9951 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:50.101569 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ebf8a9-5a39-4403-916a-4bb8709233cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:50.102192 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebf8a9-5a39-4403-916a-4bb8709233cd None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:50.102476 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebf8a9-5a39-4403-916a-4bb8709233cd None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:50.102758 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebf8a9-5a39-4403-916a-4bb8709233cd None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:50.103474 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ebf8a9-5a39-4403-916a-4bb8709233cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:50.103783 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ebf8a9-5a39-4403-916a-4bb8709233cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:50.105182 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ebf8a9-5a39-4403-916a-4bb8709233cd None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:50.113889 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 577/1181] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:57:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:50.119688 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93acda01-8b4a-4425-8e6b-282505df668e None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:50.216636 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93acda01-8b4a-4425-8e6b-282505df668e None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:50.249004 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb943e9-9206-4b01-ba6d-57e4ef2b45b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:50.250223 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb943e9-9206-4b01-ba6d-57e4ef2b45b5 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:50.250505 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb943e9-9206-4b01-ba6d-57e4ef2b45b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:50.250780 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb943e9-9206-4b01-ba6d-57e4ef2b45b5 None admin] PATH_INFO: `/v3/users/efa079a07d3e4ec9a4a46fc810587987` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:50.263643 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb943e9-9206-4b01-ba6d-57e4ef2b45b5 None admin] RBAC: Authorizing `identity:get_user(user_id=efa079a07d3e4ec9a4a46fc810587987)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:50.265119 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb943e9-9206-4b01-ba6d-57e4ef2b45b5 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:50.266726 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 605/1182] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:57:50 2022] GET /identity/v3/users/efa079a07d3e4ec9a4a46fc810587987 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:50.272521 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ebf8a9-5a39-4403-916a-4bb8709233cd None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:50.372804 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ebf8a9-5a39-4403-916a-4bb8709233cd None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:50.397163 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:50.405749 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad81ce-3975-47d8-8721-2f4b804fe529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:50.406387 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad81ce-3975-47d8-8721-2f4b804fe529 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:50.406666 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad81ce-3975-47d8-8721-2f4b804fe529 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:50.406940 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad81ce-3975-47d8-8721-2f4b804fe529 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:50.473044 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad81ce-3975-47d8-8721-2f4b804fe529 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:50.474416 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad81ce-3975-47d8-8721-2f4b804fe529 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:50.567839 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 578/1183] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:57:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 13:57:50.573991 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb943e9-9206-4b01-ba6d-57e4ef2b45b5 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:50.589182 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:50.680646 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb943e9-9206-4b01-ba6d-57e4ef2b45b5 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:50.712727 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29d26a-b02e-4d02-9c41-13f9aa288f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:50.713367 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29d26a-b02e-4d02-9c41-13f9aa288f0d None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:50.713654 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29d26a-b02e-4d02-9c41-13f9aa288f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:50.713935 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29d26a-b02e-4d02-9c41-13f9aa288f0d None admin] PATH_INFO: `/v3/projects/246d37d9b175437dbd77feb04162626b` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:50.715086 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d29d26a-b02e-4d02-9c41-13f9aa288f0d None admin] RBAC: Authorizing `identity:get_project(project_id=246d37d9b175437dbd77feb04162626b)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:50.716525 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d29d26a-b02e-4d02-9c41-13f9aa288f0d None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:50.718182 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 606/1184] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:57:50 2022] GET /identity/v3/projects/246d37d9b175437dbd77feb04162626b => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:50.724867 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad81ce-3975-47d8-8721-2f4b804fe529 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:50.751220 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:50.822719 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad81ce-3975-47d8-8721-2f4b804fe529 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:50.855104 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958da250-3790-413a-91c1-6666799bbc1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:50.855710 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958da250-3790-413a-91c1-6666799bbc1a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:50.855993 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958da250-3790-413a-91c1-6666799bbc1a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:50.856271 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958da250-3790-413a-91c1-6666799bbc1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:50.857081 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958da250-3790-413a-91c1-6666799bbc1a None admin] RBAC: Adding query filter params (role.id=58f91e3a13664dabb18c6227a1658fa6, scope.project.id=246d37d9b175437dbd77feb04162626b, user.id=efa079a07d3e4ec9a4a46fc810587987) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:50.857391 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958da250-3790-413a-91c1-6666799bbc1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:50.858791 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958da250-3790-413a-91c1-6666799bbc1a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:50.864670 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 579/1185] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:57:50 2022] GET /identity/v3/role_assignments?user.id=efa079a07d3e4ec9a4a46fc810587987&scope.project.id=246d37d9b175437dbd77feb04162626b&role.id=58f91e3a13664dabb18c6227a1658fa6 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:52.817056 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b92e23-b897-4269-bac3-d0a591bf4dea None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:52.817509 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b92e23-b897-4269-bac3-d0a591bf4dea None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:52.817509 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b92e23-b897-4269-bac3-d0a591bf4dea None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:52.818785 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 607/1186] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:57:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:57:52.824020 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152908e-1848-4060-bb3c-234728689dd5 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:52.824403 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152908e-1848-4060-bb3c-234728689dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:52.824615 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152908e-1848-4060-bb3c-234728689dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:52.843055 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8152908e-1848-4060-bb3c-234728689dd5 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:57:52.844245 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8152908e-1848-4060-bb3c-234728689dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:57:52.909560 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 580/1187] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:57:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:57:52.931634 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b92e23-b897-4269-bac3-d0a591bf4dea None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:53.021664 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:53.040746 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b92e23-b897-4269-bac3-d0a591bf4dea None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:53.074069 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e7ba6-5220-42fa-be8d-655eaad1fc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:53.074630 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e7ba6-5220-42fa-be8d-655eaad1fc51 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:53.074908 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e7ba6-5220-42fa-be8d-655eaad1fc51 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:53.075177 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e7ba6-5220-42fa-be8d-655eaad1fc51 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:53.085353 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1e7ba6-5220-42fa-be8d-655eaad1fc51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:57:53.090109 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-5a1e7ba6-5220-42fa-be8d-655eaad1fc51 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:57:53.090109 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:53.090109 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:53.090109 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:53.090109 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:53.090109 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:53.090109 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:53.090109 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:53.090109 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:53.090109 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:53.090109 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:53.090109 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:53.090109 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:57:53.090109 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:57:53.090109 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:57:53.090109 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:57:53.091201 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:57:53.091201 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:57:53.091201 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:57:53.091201 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:57:53.091201 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:53.091201 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:53.091201 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:53.091201 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:53.091201 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:53.091201 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:53.091201 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:53.091201 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:53.091201 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:53.091201 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:53.091201 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:53.091201 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:53.092409 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:53.092409 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:53.092409 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:53.092409 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:53.092409 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:57:53.092409 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:57:53.092409 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:57:53.092409 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:57:53.092409 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:57:53.092409 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:57:53.092409 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:57:53.092409 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:57:53.092409 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 608/1188] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 13:57:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:53.097345 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8152908e-1848-4060-bb3c-234728689dd5 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:53.117515 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:53.193943 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8152908e-1848-4060-bb3c-234728689dd5 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:53.226178 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a6e325-4c4d-4547-ac9e-e165197e02f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:53.226791 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a6e325-4c4d-4547-ac9e-e165197e02f2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:53.227076 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a6e325-4c4d-4547-ac9e-e165197e02f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:53.227351 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a6e325-4c4d-4547-ac9e-e165197e02f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:53.228059 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a6e325-4c4d-4547-ac9e-e165197e02f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:53.228365 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a6e325-4c4d-4547-ac9e-e165197e02f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:53.229749 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a6e325-4c4d-4547-ac9e-e165197e02f2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:53.238089 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 581/1189] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:57:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:53.243993 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e7ba6-5220-42fa-be8d-655eaad1fc51 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:53.343809 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e7ba6-5220-42fa-be8d-655eaad1fc51 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:53.376048 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93485be-4d1d-4c1e-98b9-d466f8d70167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:53.376951 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93485be-4d1d-4c1e-98b9-d466f8d70167 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:53.377230 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93485be-4d1d-4c1e-98b9-d466f8d70167 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:53.377500 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93485be-4d1d-4c1e-98b9-d466f8d70167 None admin] PATH_INFO: `/v3/users/efa079a07d3e4ec9a4a46fc810587987` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:53.378644 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93485be-4d1d-4c1e-98b9-d466f8d70167 None admin] RBAC: Authorizing `identity:get_user(user_id=efa079a07d3e4ec9a4a46fc810587987)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:53.380061 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93485be-4d1d-4c1e-98b9-d466f8d70167 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:53.381666 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 609/1190] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:57:53 2022] GET /identity/v3/users/efa079a07d3e4ec9a4a46fc810587987 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:53.387485 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a6e325-4c4d-4547-ac9e-e165197e02f2 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:53.466760 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:53.485046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a6e325-4c4d-4547-ac9e-e165197e02f2 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:53.519493 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b36be3-ab7a-4d57-b273-1e0449fb69ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:53.520279 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b36be3-ab7a-4d57-b273-1e0449fb69ae None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:53.521674 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b36be3-ab7a-4d57-b273-1e0449fb69ae None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:53.521970 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b36be3-ab7a-4d57-b273-1e0449fb69ae None admin] PATH_INFO: `/v3/projects/246d37d9b175437dbd77feb04162626b` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:53.523155 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b36be3-ab7a-4d57-b273-1e0449fb69ae None admin] RBAC: Authorizing `identity:get_project(project_id=246d37d9b175437dbd77feb04162626b)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:53.524648 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b36be3-ab7a-4d57-b273-1e0449fb69ae None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:53.526315 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 582/1191] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:57:53 2022] GET /identity/v3/projects/246d37d9b175437dbd77feb04162626b => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:53.532891 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93485be-4d1d-4c1e-98b9-d466f8d70167 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:53.545325 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:53.631263 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93485be-4d1d-4c1e-98b9-d466f8d70167 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:53.663651 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8d56a-9c5f-40e1-be12-a13bdd1c1b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:53.664321 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8d56a-9c5f-40e1-be12-a13bdd1c1b91 None admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:53.664629 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8d56a-9c5f-40e1-be12-a13bdd1c1b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:53.664909 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8d56a-9c5f-40e1-be12-a13bdd1c1b91 None admin] PATH_INFO: `/v3/projects/246d37d9b175437dbd77feb04162626b/users/efa079a07d3e4ec9a4a46fc810587987/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:53.666809 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a8d56a-9c5f-40e1-be12-a13bdd1c1b91 None admin] RBAC: Authorizing `identity:create_grant(project_id=246d37d9b175437dbd77feb04162626b, user_id=efa079a07d3e4ec9a4a46fc810587987, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:53.668235 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a8d56a-9c5f-40e1-be12-a13bdd1c1b91 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:53.682120 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 610/1192] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 13:57:53 2022] PUT /identity/v3/projects/246d37d9b175437dbd77feb04162626b/users/efa079a07d3e4ec9a4a46fc810587987/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:57:55.633851 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643cbb0d-589a-4d78-8b80-a995edf0ee88 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:55.634400 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643cbb0d-589a-4d78-8b80-a995edf0ee88 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:55.634400 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643cbb0d-589a-4d78-8b80-a995edf0ee88 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:55.635748 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 583/1193] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:57:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:57:55.641647 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e8cc2-590e-4336-9799-af84068bc19f None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:55.642048 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e8cc2-590e-4336-9799-af84068bc19f None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:55.642205 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e8cc2-590e-4336-9799-af84068bc19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:55.660404 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-906e8cc2-590e-4336-9799-af84068bc19f None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:57:55.661673 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-906e8cc2-590e-4336-9799-af84068bc19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:57:55.722256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:55.727703 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 611/1194] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:57:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:57:55.750376 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643cbb0d-589a-4d78-8b80-a995edf0ee88 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:55.851676 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643cbb0d-589a-4d78-8b80-a995edf0ee88 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:55.892649 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4a66a-1e27-43e2-985a-aa2f8525a1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:55.893219 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4a66a-1e27-43e2-985a-aa2f8525a1b2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:55.893505 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4a66a-1e27-43e2-985a-aa2f8525a1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:55.893784 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4a66a-1e27-43e2-985a-aa2f8525a1b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:55.904424 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac4a66a-1e27-43e2-985a-aa2f8525a1b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:57:55.909363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-0ac4a66a-1e27-43e2-985a-aa2f8525a1b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:57:55.909363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:57:55.909363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:57:55.909363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:57:55.909363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:57:55.909363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:57:55.909363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:57:55.909363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:57:55.909363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:57:55.909363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:57:55.909363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:57:55.909363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:57:55.909363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:57:55.909363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:57:55.909363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:57:55.909363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:57:55.910440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:57:55.910440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:57:55.910440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:57:55.910440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:57:55.910440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:57:55.910440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:57:55.910440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:57:55.910440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:57:55.910440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:57:55.910440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:57:55.910440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:57:55.910440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:57:55.910440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:57:55.910440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:57:55.910440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:57:55.910440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:57:55.911441 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:57:55.911441 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:57:55.911441 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:57:55.911441 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:57:55.911441 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:57:55.911441 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:57:55.911441 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:57:55.911441 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:57:55.911441 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:57:55.911441 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:57:55.911441 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:57:55.911441 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:57:55.911441 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 584/1195] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 13:57:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:57:55.916790 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e8cc2-590e-4336-9799-af84068bc19f None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:56.013483 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e8cc2-590e-4336-9799-af84068bc19f None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:56.046042 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b830e2f-eae2-4155-b463-1c6fdbf65b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:56.046654 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b830e2f-eae2-4155-b463-1c6fdbf65b94 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:56.046936 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b830e2f-eae2-4155-b463-1c6fdbf65b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:56.047211 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b830e2f-eae2-4155-b463-1c6fdbf65b94 None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:56.047924 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b830e2f-eae2-4155-b463-1c6fdbf65b94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:56.048229 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b830e2f-eae2-4155-b463-1c6fdbf65b94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:56.049626 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b830e2f-eae2-4155-b463-1c6fdbf65b94 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:56.057342 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 612/1196] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:57:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:56.063705 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4a66a-1e27-43e2-985a-aa2f8525a1b2 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:56.076020 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:56.160835 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4a66a-1e27-43e2-985a-aa2f8525a1b2 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:56.194419 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f4bfe2-e319-4715-a427-9a1c7c60d8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:56.195063 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4bfe2-e319-4715-a427-9a1c7c60d8af None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:56.195348 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4bfe2-e319-4715-a427-9a1c7c60d8af None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:56.195624 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4bfe2-e319-4715-a427-9a1c7c60d8af None admin] PATH_INFO: `/v3/users/efa079a07d3e4ec9a4a46fc810587987` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:56.196810 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f4bfe2-e319-4715-a427-9a1c7c60d8af None admin] RBAC: Authorizing `identity:get_user(user_id=efa079a07d3e4ec9a4a46fc810587987)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:56.198256 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f4bfe2-e319-4715-a427-9a1c7c60d8af None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:56.199852 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 585/1197] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 13:57:56 2022] GET /identity/v3/users/efa079a07d3e4ec9a4a46fc810587987 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:56.205779 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b830e2f-eae2-4155-b463-1c6fdbf65b94 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:56.278699 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:56.317483 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b830e2f-eae2-4155-b463-1c6fdbf65b94 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:56.350441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564aa782-31c9-45f9-aaf2-5a640b452c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:56.351058 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564aa782-31c9-45f9-aaf2-5a640b452c90 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:56.351343 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564aa782-31c9-45f9-aaf2-5a640b452c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:56.351623 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564aa782-31c9-45f9-aaf2-5a640b452c90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:56.416952 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564aa782-31c9-45f9-aaf2-5a640b452c90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:56.418329 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564aa782-31c9-45f9-aaf2-5a640b452c90 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:56.476282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:56.514976 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 613/1198] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:57:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 13:57:56.520859 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f4bfe2-e319-4715-a427-9a1c7c60d8af None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:56.595687 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:57:56.616775 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f4bfe2-e319-4715-a427-9a1c7c60d8af None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:56.648258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a1dfb7-1cf8-432a-a4db-dcbafcc37b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:56.648915 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1dfb7-1cf8-432a-a4db-dcbafcc37b06 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:56.649193 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1dfb7-1cf8-432a-a4db-dcbafcc37b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:56.649482 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1dfb7-1cf8-432a-a4db-dcbafcc37b06 None admin] PATH_INFO: `/v3/projects/246d37d9b175437dbd77feb04162626b` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:56.650600 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a1dfb7-1cf8-432a-a4db-dcbafcc37b06 None admin] RBAC: Authorizing `identity:get_project(project_id=246d37d9b175437dbd77feb04162626b)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:56.652003 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a1dfb7-1cf8-432a-a4db-dcbafcc37b06 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:56.653653 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 586/1199] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 13:57:56 2022] GET /identity/v3/projects/246d37d9b175437dbd77feb04162626b => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:57:56.660205 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564aa782-31c9-45f9-aaf2-5a640b452c90 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:57:56.756038 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564aa782-31c9-45f9-aaf2-5a640b452c90 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:57:56.788616 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0bee02-dd29-46e8-b91d-b323643d5b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:57:56.789221 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0bee02-dd29-46e8-b91d-b323643d5b50 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:57:56.789503 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0bee02-dd29-46e8-b91d-b323643d5b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:57:56.789781 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0bee02-dd29-46e8-b91d-b323643d5b50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:57:56.790564 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0bee02-dd29-46e8-b91d-b323643d5b50 None admin] RBAC: Adding query filter params (role.id=58f91e3a13664dabb18c6227a1658fa6, scope.project.id=246d37d9b175437dbd77feb04162626b, user.id=efa079a07d3e4ec9a4a46fc810587987) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:57:56.790872 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0bee02-dd29-46e8-b91d-b323643d5b50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:57:56.792281 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0bee02-dd29-46e8-b91d-b323643d5b50 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:57:56.798517 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 614/1200] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:57:56 2022] GET /identity/v3/role_assignments?user.id=efa079a07d3e4ec9a4a46fc810587987&scope.project.id=246d37d9b175437dbd77feb04162626b&role.id=58f91e3a13664dabb18c6227a1658fa6 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:58:06.644343 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=27 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:58:06.795414 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=87 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:58:51.710947 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732caecc-d53f-4f3a-861b-a9cc51c13db2 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:51.710947 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732caecc-d53f-4f3a-861b-a9cc51c13db2 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:51.710947 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732caecc-d53f-4f3a-861b-a9cc51c13db2 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:51.712068 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 587/1201] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:58:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 13:58:51.718472 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584504d-808e-4264-b2fe-8e3e44e1f6ca None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:51.719287 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584504d-808e-4264-b2fe-8e3e44e1f6ca None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:51.719595 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584504d-808e-4264-b2fe-8e3e44e1f6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:51.745228 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-0584504d-808e-4264-b2fe-8e3e44e1f6ca None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:58:51.746836 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-0584504d-808e-4264-b2fe-8e3e44e1f6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:58:51.828530 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 615/1202] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:58:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:58:51.850483 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732caecc-d53f-4f3a-861b-a9cc51c13db2 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:51.976548 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732caecc-d53f-4f3a-861b-a9cc51c13db2 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:52.020066 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a314b4-c5f6-4def-b759-4ab7ce32797a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:52.020739 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a314b4-c5f6-4def-b759-4ab7ce32797a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:52.021021 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a314b4-c5f6-4def-b759-4ab7ce32797a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:52.021304 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a314b4-c5f6-4def-b759-4ab7ce32797a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:52.035798 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a314b4-c5f6-4def-b759-4ab7ce32797a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:58:52.037403 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a314b4-c5f6-4def-b759-4ab7ce32797a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:58:52.054523 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-34a314b4-c5f6-4def-b759-4ab7ce32797a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:58:52.054523 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:58:52.054523 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:58:52.054523 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:58:52.054523 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:58:52.054523 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:58:52.054523 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:58:52.054523 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:58:52.054523 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:58:52.054523 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:58:52.054523 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:58:52.054523 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:58:52.054523 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:58:52.054523 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:58:52.054523 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:58:52.056169 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:58:52.056169 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:58:52.056169 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:58:52.056169 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:58:52.056169 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:58:52.056169 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:58:52.056169 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:58:52.056169 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:58:52.056169 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:58:52.056169 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:58:52.056169 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:58:52.056169 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:58:52.056169 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:58:52.056169 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:58:52.056169 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:58:52.056169 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:58:52.056169 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:58:52.057759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:58:52.057759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:58:52.057759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:58:52.057759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:58:52.061687 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 588/1203] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:58:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:58:52.062984 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0584504d-808e-4264-b2fe-8e3e44e1f6ca None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:52.195401 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0584504d-808e-4264-b2fe-8e3e44e1f6ca None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:52.234551 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:58:52.241947 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80110339-58fe-4b19-86ec-424d8983f378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:52.242775 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80110339-58fe-4b19-86ec-424d8983f378 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:52.243139 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80110339-58fe-4b19-86ec-424d8983f378 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:52.243352 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80110339-58fe-4b19-86ec-424d8983f378 None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:52.244289 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80110339-58fe-4b19-86ec-424d8983f378 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:58:52.244636 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80110339-58fe-4b19-86ec-424d8983f378 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:58:52.248536 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80110339-58fe-4b19-86ec-424d8983f378 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:58:52.260524 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 616/1204] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:58:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:58:52.266652 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a314b4-c5f6-4def-b759-4ab7ce32797a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:52.359675 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:58:52.445944 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a314b4-c5f6-4def-b759-4ab7ce32797a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:52.488438 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b50b2d-5e34-4204-b16e-d30c9f0d4221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:52.489027 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b50b2d-5e34-4204-b16e-d30c9f0d4221 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:52.489317 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b50b2d-5e34-4204-b16e-d30c9f0d4221 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:52.489599 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b50b2d-5e34-4204-b16e-d30c9f0d4221 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:52.490301 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b50b2d-5e34-4204-b16e-d30c9f0d4221 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:58:52.491793 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b50b2d-5e34-4204-b16e-d30c9f0d4221 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:58:52.528509 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 589/1205] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 13:58:52 2022] POST /identity/v3/users => generated 251 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:58:54.747866 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fe57a-55ee-4e6f-8642-385751640295 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:54.748405 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fe57a-55ee-4e6f-8642-385751640295 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:54.748536 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fe57a-55ee-4e6f-8642-385751640295 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:54.750425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 617/1206] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:58:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:58:54.755911 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901cbbf5-c426-4696-bff1-9aeb054d69d5 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:54.756337 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901cbbf5-c426-4696-bff1-9aeb054d69d5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:54.756505 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901cbbf5-c426-4696-bff1-9aeb054d69d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:54.776812 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-901cbbf5-c426-4696-bff1-9aeb054d69d5 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:58:54.778640 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-901cbbf5-c426-4696-bff1-9aeb054d69d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:58:54.861452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 590/1207] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:58:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:58:54.887071 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0fe57a-55ee-4e6f-8642-385751640295 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:55.030439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0fe57a-55ee-4e6f-8642-385751640295 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:55.070817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e7f84d-251e-45cd-9286-9c6b8bd900e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:55.071659 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7f84d-251e-45cd-9286-9c6b8bd900e7 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:55.072185 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7f84d-251e-45cd-9286-9c6b8bd900e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:55.072460 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7f84d-251e-45cd-9286-9c6b8bd900e7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:55.088000 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e7f84d-251e-45cd-9286-9c6b8bd900e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:58:55.095426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-c5e7f84d-251e-45cd-9286-9c6b8bd900e7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:58:55.095426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:58:55.095426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:58:55.095426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:58:55.095426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:58:55.095426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:58:55.095426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:58:55.095426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:58:55.095426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:58:55.095426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:58:55.095426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:58:55.095426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:58:55.095426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:58:55.095426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:58:55.095426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:58:55.095426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:58:55.096645 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:58:55.096645 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:58:55.096645 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:58:55.096645 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:58:55.096645 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:58:55.096645 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:58:55.096645 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:58:55.096645 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:58:55.096645 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:58:55.096645 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:58:55.096645 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:58:55.096645 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:58:55.096645 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:58:55.096645 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:58:55.096645 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:58:55.096645 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:58:55.101461 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:58:55.101461 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:58:55.101461 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:58:55.101461 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:58:55.101461 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:58:55.101461 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:58:55.101461 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:58:55.101461 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:58:55.101461 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:58:55.101461 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:58:55.101461 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:58:55.101461 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:58:55.101461 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 618/1208] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 13:58:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:58:55.103154 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901cbbf5-c426-4696-bff1-9aeb054d69d5 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:55.147329 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:58:55.222718 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901cbbf5-c426-4696-bff1-9aeb054d69d5 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:55.267263 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975493e-3514-448e-a5db-97f3988fbbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:55.268546 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975493e-3514-448e-a5db-97f3988fbbf1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:55.268915 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975493e-3514-448e-a5db-97f3988fbbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:55.269301 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975493e-3514-448e-a5db-97f3988fbbf1 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:55.270038 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5975493e-3514-448e-a5db-97f3988fbbf1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:58:55.270411 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5975493e-3514-448e-a5db-97f3988fbbf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:58:55.272125 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5975493e-3514-448e-a5db-97f3988fbbf1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:58:55.289925 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 591/1209] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 13:58:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:58:55.295079 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e7f84d-251e-45cd-9286-9c6b8bd900e7 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:55.360990 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:58:55.428651 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e7f84d-251e-45cd-9286-9c6b8bd900e7 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:55.465862 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd5eed-1156-48d1-8e3e-e2544c967c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:55.466545 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd5eed-1156-48d1-8e3e-e2544c967c8f None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:55.466825 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd5eed-1156-48d1-8e3e-e2544c967c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:55.467105 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd5eed-1156-48d1-8e3e-e2544c967c8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:55.478696 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bd5eed-1156-48d1-8e3e-e2544c967c8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:58:55.480298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bd5eed-1156-48d1-8e3e-e2544c967c8f None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:58:55.496520 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-35bd5eed-1156-48d1-8e3e-e2544c967c8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:58:55.496520 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:58:55.496520 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:58:55.496520 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:58:55.496520 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:58:55.496520 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:58:55.496520 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:58:55.496520 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:58:55.496520 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:58:55.496520 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:58:55.496520 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:58:55.496520 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:58:55.496520 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:58:55.496520 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:58:55.496520 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:58:55.497668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:58:55.497668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:58:55.497668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:58:55.497668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:58:55.497668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:58:55.497668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:58:55.497668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:58:55.497668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:58:55.497668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:58:55.497668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:58:55.497668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:58:55.497668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:58:55.497668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:58:55.497668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:58:55.497668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:58:55.497668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:58:55.497668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:58:55.499010 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:58:55.499010 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:58:55.499010 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:58:55.499010 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:58:55.500628 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 619/1210] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:58:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:58:55.503632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975493e-3514-448e-a5db-97f3988fbbf1 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:55.630247 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:58:55.634187 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975493e-3514-448e-a5db-97f3988fbbf1 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:55.673434 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f3ea4-08ab-43c9-9a41-89370c3f3a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:55.674164 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f3ea4-08ab-43c9-9a41-89370c3f3a3e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:55.674553 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f3ea4-08ab-43c9-9a41-89370c3f3a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:55.674929 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f3ea4-08ab-43c9-9a41-89370c3f3a3e None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:55.675742 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f3ea4-08ab-43c9-9a41-89370c3f3a3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:58:55.676148 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f3ea4-08ab-43c9-9a41-89370c3f3a3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:58:55.677735 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f3ea4-08ab-43c9-9a41-89370c3f3a3e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:58:55.691684 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 592/1211] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:58:55.697513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd5eed-1156-48d1-8e3e-e2544c967c8f None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:55.830082 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd5eed-1156-48d1-8e3e-e2544c967c8f None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:55.831942 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:58:55.871742 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3343b127-23d6-422f-8406-eefd7d27e8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:55.872404 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343b127-23d6-422f-8406-eefd7d27e8d0 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:55.872723 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343b127-23d6-422f-8406-eefd7d27e8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:55.873005 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343b127-23d6-422f-8406-eefd7d27e8d0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:55.882139 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3343b127-23d6-422f-8406-eefd7d27e8d0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:58:55.883636 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3343b127-23d6-422f-8406-eefd7d27e8d0 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:58:55.896378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-3343b127-23d6-422f-8406-eefd7d27e8d0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 13:58:55.896378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:58:55.896378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:58:55.896378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:58:55.896378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:58:55.896378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:58:55.896378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:58:55.896378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:58:55.896378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:58:55.896378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:58:55.896378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:58:55.896378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:58:55.896378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:58:55.896378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:58:55.896378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:58:55.896378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:58:55.897390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:58:55.897390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:58:55.897390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:58:55.897390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:58:55.897390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:58:55.897390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:58:55.897390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:58:55.897390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:58:55.897390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:58:55.897390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:58:55.897390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:58:55.897390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:58:55.897390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:58:55.897390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:58:55.897390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:58:55.897390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:58:55.897390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:58:55.897390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:58:55.898317 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:58:55.898317 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:58:55.898317 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:58:55.898317 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:58:55.898317 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:58:55.898317 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:58:55.898317 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:58:55.898317 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:58:55.898317 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 13:58:55.898317 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:58:55.900248 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 620/1212] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:58:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:58:55.907564 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f3ea4-08ab-43c9-9a41-89370c3f3a3e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:56.038483 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f3ea4-08ab-43c9-9a41-89370c3f3a3e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:56.078397 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418c79a-0609-4d71-abd6-eabc0f1af4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:56.079091 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418c79a-0609-4d71-abd6-eabc0f1af4e1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:56.079399 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418c79a-0609-4d71-abd6-eabc0f1af4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:56.079757 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418c79a-0609-4d71-abd6-eabc0f1af4e1 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:56.080688 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b418c79a-0609-4d71-abd6-eabc0f1af4e1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:58:56.081005 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b418c79a-0609-4d71-abd6-eabc0f1af4e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:58:56.083697 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b418c79a-0609-4d71-abd6-eabc0f1af4e1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:58:56.107610 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 593/1213] 10.222.0.48 () {56 vars in 1143 bytes} [Mon Aug 1 13:58:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:58:56.108859 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3343b127-23d6-422f-8406-eefd7d27e8d0 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:56.232653 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3343b127-23d6-422f-8406-eefd7d27e8d0 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:56.273318 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17072ecd-62c4-405c-80fe-7dbd343fd29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:56.274016 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17072ecd-62c4-405c-80fe-7dbd343fd29f None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:56.274370 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17072ecd-62c4-405c-80fe-7dbd343fd29f None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:56.274671 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17072ecd-62c4-405c-80fe-7dbd343fd29f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:56.309264 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:58:56.359450 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17072ecd-62c4-405c-80fe-7dbd343fd29f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:58:56.360944 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17072ecd-62c4-405c-80fe-7dbd343fd29f None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:58:56.475911 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 621/1214] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:58:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 13:58:56.481677 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418c79a-0609-4d71-abd6-eabc0f1af4e1 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:56.503101 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:58:56.597203 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418c79a-0609-4d71-abd6-eabc0f1af4e1 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:56.634793 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f54e09-4874-483e-a619-2093890254cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:56.635500 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f54e09-4874-483e-a619-2093890254cb None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:56.635790 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f54e09-4874-483e-a619-2093890254cb None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:56.636067 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f54e09-4874-483e-a619-2093890254cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:56.647993 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f54e09-4874-483e-a619-2093890254cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:58:56.649493 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f54e09-4874-483e-a619-2093890254cb None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:58:56.666955 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-b3f54e09-4874-483e-a619-2093890254cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:58:56.666955 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:58:56.666955 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:58:56.666955 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:58:56.666955 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:58:56.666955 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:58:56.666955 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:58:56.666955 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:58:56.666955 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:58:56.666955 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:58:56.666955 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:58:56.666955 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:58:56.666955 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:58:56.666955 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:58:56.666955 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:58:56.668465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:58:56.668465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:58:56.668465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:58:56.668465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:58:56.668465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:58:56.668465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:58:56.668465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:58:56.668465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:58:56.668465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:58:56.668465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:58:56.668465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:58:56.668465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:58:56.668465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:58:56.668465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:58:56.668465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:58:56.668465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:58:56.668465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:58:56.670073 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:58:56.670073 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:58:56.670073 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:58:56.670073 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:58:56.670073 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 594/1215] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:58:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:58:56.674795 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17072ecd-62c4-405c-80fe-7dbd343fd29f None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:56.751694 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:58:56.795027 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17072ecd-62c4-405c-80fe-7dbd343fd29f None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:56.836510 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa2bbf4-ab64-4021-a473-3cc20b5c574f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:56.837201 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2bbf4-ab64-4021-a473-3cc20b5c574f None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:56.837472 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2bbf4-ab64-4021-a473-3cc20b5c574f None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:56.837723 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2bbf4-ab64-4021-a473-3cc20b5c574f None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:56.838709 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa2bbf4-ab64-4021-a473-3cc20b5c574f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:58:56.839185 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa2bbf4-ab64-4021-a473-3cc20b5c574f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:58:56.841506 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa2bbf4-ab64-4021-a473-3cc20b5c574f None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:58:56.854234 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 622/1216] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:58:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:58:56.859797 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f54e09-4874-483e-a619-2093890254cb None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:56.943482 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:58:56.978693 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f54e09-4874-483e-a619-2093890254cb None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:57.014424 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeab754-4785-45ea-912a-00dfb1182d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:57.015090 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeab754-4785-45ea-912a-00dfb1182d49 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:57.015392 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeab754-4785-45ea-912a-00dfb1182d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:57.015711 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeab754-4785-45ea-912a-00dfb1182d49 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:57.023131 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeab754-4785-45ea-912a-00dfb1182d49 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:58:57.024700 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeab754-4785-45ea-912a-00dfb1182d49 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:58:57.038036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-baeab754-4785-45ea-912a-00dfb1182d49 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:58:57.038036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:58:57.038036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:58:57.038036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:58:57.038036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:58:57.038036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:58:57.038036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:58:57.038036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:58:57.038036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:58:57.038036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:58:57.038036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:58:57.038036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:58:57.038036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:58:57.038036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:58:57.038036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:58:57.040013 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:58:57.040013 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:58:57.040013 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:58:57.040013 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:58:57.040013 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:58:57.040013 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:58:57.040013 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:58:57.040013 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:58:57.040013 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:58:57.040013 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:58:57.040013 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:58:57.040013 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:58:57.040013 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:58:57.040013 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:58:57.040013 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:58:57.040013 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:58:57.040013 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:58:57.041279 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:58:57.041279 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:58:57.041279 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:58:57.041279 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:58:57.041279 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:58:57.041279 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:58:57.041279 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:58:57.041279 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:58:57.041279 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 595/1217] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:58:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:58:57.046719 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa2bbf4-ab64-4021-a473-3cc20b5c574f None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:57.164099 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa2bbf4-ab64-4021-a473-3cc20b5c574f None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:57.199761 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:58:57.212376 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d62c5-dca3-440a-bac0-af73b875a540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:57.213182 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d62c5-dca3-440a-bac0-af73b875a540 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:57.213564 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d62c5-dca3-440a-bac0-af73b875a540 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:57.214190 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d62c5-dca3-440a-bac0-af73b875a540 None admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:57.215735 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d62c5-dca3-440a-bac0-af73b875a540 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:58:57.216273 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d62c5-dca3-440a-bac0-af73b875a540 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:58:57.218335 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d62c5-dca3-440a-bac0-af73b875a540 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:58:57.237533 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 623/1218] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 13:58:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:58:57.239068 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeab754-4785-45ea-912a-00dfb1182d49 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:57.366828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeab754-4785-45ea-912a-00dfb1182d49 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:57.398255 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:58:57.406053 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857b7b0-9d34-4ceb-9a14-e30aab02079a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:57.406742 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857b7b0-9d34-4ceb-9a14-e30aab02079a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:57.407092 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857b7b0-9d34-4ceb-9a14-e30aab02079a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:57.407435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857b7b0-9d34-4ceb-9a14-e30aab02079a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:57.408288 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f857b7b0-9d34-4ceb-9a14-e30aab02079a None admin] RBAC: Adding query filter params (role.id=d3d8169e71454005a8cf0d7544ad05dc, scope.project.id=72ecec9fad964b1fbeaefd0519597229, user.id=e294fe796a1a4ac89041a532856ad627) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:58:57.408733 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f857b7b0-9d34-4ceb-9a14-e30aab02079a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:58:57.410262 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f857b7b0-9d34-4ceb-9a14-e30aab02079a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:58:57.416031 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 596/1219] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:58:57 2022] GET /identity/v3/role_assignments?user.id=e294fe796a1a4ac89041a532856ad627&scope.project.id=72ecec9fad964b1fbeaefd0519597229&role.id=d3d8169e71454005a8cf0d7544ad05dc => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:58:59.414095 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b54c2-5dbf-440b-a62c-ab18044e8dfd None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:59.414657 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b54c2-5dbf-440b-a62c-ab18044e8dfd None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:59.414657 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b54c2-5dbf-440b-a62c-ab18044e8dfd None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:59.416044 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 624/1220] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:58:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:58:59.421343 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928c5db-e367-47b1-adfd-837bbc5623a0 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:59.421763 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928c5db-e367-47b1-adfd-837bbc5623a0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:59.421906 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928c5db-e367-47b1-adfd-837bbc5623a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:59.441081 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-0928c5db-e367-47b1-adfd-837bbc5623a0 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:58:59.442467 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0928c5db-e367-47b1-adfd-837bbc5623a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:58:59.518500 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 597/1221] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:58:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:58:59.546174 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b54c2-5dbf-440b-a62c-ab18044e8dfd None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:59.662741 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b54c2-5dbf-440b-a62c-ab18044e8dfd None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:59.704430 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543e753-0f90-4e78-b186-940e00e84fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:59.705032 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543e753-0f90-4e78-b186-940e00e84fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:59.705315 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543e753-0f90-4e78-b186-940e00e84fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:59.705704 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543e753-0f90-4e78-b186-940e00e84fe4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:59.721494 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a543e753-0f90-4e78-b186-940e00e84fe4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:58:59.731155 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-a543e753-0f90-4e78-b186-940e00e84fe4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:58:59.731155 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:58:59.731155 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:58:59.731155 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:58:59.731155 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:58:59.731155 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:58:59.731155 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:58:59.731155 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:58:59.731155 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:58:59.731155 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:58:59.731155 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:58:59.731155 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:58:59.731155 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:58:59.731155 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:58:59.731155 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:58:59.731155 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:58:59.732296 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:58:59.732296 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:58:59.732296 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:58:59.732296 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:58:59.732296 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:58:59.732296 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:58:59.732296 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:58:59.732296 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:58:59.732296 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:58:59.732296 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:58:59.732296 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:58:59.732296 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:58:59.732296 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:58:59.732296 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:58:59.732296 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:58:59.732296 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:58:59.733246 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:58:59.733246 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:58:59.733246 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:58:59.733246 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:58:59.733246 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:58:59.733246 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:58:59.733246 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:58:59.733246 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:58:59.733246 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:58:59.733246 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:58:59.733246 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:58:59.733246 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:58:59.733246 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 625/1222] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 13:58:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:58:59.742645 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0928c5db-e367-47b1-adfd-837bbc5623a0 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:59.859103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0928c5db-e367-47b1-adfd-837bbc5623a0 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:58:59.887150 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:58:59.898442 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e745ca4-484e-4952-b829-6aaa60ee3271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:58:59.899069 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e745ca4-484e-4952-b829-6aaa60ee3271 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:58:59.899356 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e745ca4-484e-4952-b829-6aaa60ee3271 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:58:59.899636 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e745ca4-484e-4952-b829-6aaa60ee3271 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:58:59.900362 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e745ca4-484e-4952-b829-6aaa60ee3271 None admin] RBAC: Adding query filter params (name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:58:59.900697 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e745ca4-484e-4952-b829-6aaa60ee3271 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:58:59.902150 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e745ca4-484e-4952-b829-6aaa60ee3271 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:58:59.910335 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 598/1223] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 13:58:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:58:59.916203 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543e753-0f90-4e78-b186-940e00e84fe4 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:58:59.962989 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:00.025808 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543e753-0f90-4e78-b186-940e00e84fe4 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:00.068119 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e3cda2-9b4c-4354-944a-24fd59929530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:00.069247 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3cda2-9b4c-4354-944a-24fd59929530 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:00.069726 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3cda2-9b4c-4354-944a-24fd59929530 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:00.070199 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3cda2-9b4c-4354-944a-24fd59929530 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:00.083008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e3cda2-9b4c-4354-944a-24fd59929530 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:00.085284 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e3cda2-9b4c-4354-944a-24fd59929530 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:00.106185 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-c3e3cda2-9b4c-4354-944a-24fd59929530 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:59:00.106185 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:59:00.106185 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:59:00.106185 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:59:00.106185 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:59:00.106185 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:59:00.106185 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:59:00.106185 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:59:00.106185 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:59:00.106185 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:59:00.106185 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:59:00.106185 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:59:00.106185 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:59:00.106185 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:59:00.106185 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:59:00.107170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:59:00.107170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:59:00.107170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:59:00.107170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:59:00.107170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:59:00.107170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:59:00.107170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:59:00.107170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:59:00.107170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:59:00.107170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:59:00.107170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:59:00.107170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:59:00.107170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:59:00.107170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:59:00.107170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:59:00.107170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:59:00.107170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:59:00.108056 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:59:00.108056 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:59:00.108056 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:59:00.108056 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:59:00.108468 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 626/1224] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:58:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 13:59:00.117573 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e745ca4-484e-4952-b829-6aaa60ee3271 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:00.228176 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e745ca4-484e-4952-b829-6aaa60ee3271 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:00.265398 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee295b-ede7-4c6e-9ce1-aa2cfca6b582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:00.266078 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee295b-ede7-4c6e-9ce1-aa2cfca6b582 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:00.266367 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee295b-ede7-4c6e-9ce1-aa2cfca6b582 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:00.266648 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee295b-ede7-4c6e-9ce1-aa2cfca6b582 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:00.267340 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ee295b-ede7-4c6e-9ce1-aa2cfca6b582 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:59:00.267648 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ee295b-ede7-4c6e-9ce1-aa2cfca6b582 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:00.269052 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ee295b-ede7-4c6e-9ce1-aa2cfca6b582 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:00.281262 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 599/1225] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:59:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:59:00.287046 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e3cda2-9b4c-4354-944a-24fd59929530 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:00.385353 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:00.393086 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e3cda2-9b4c-4354-944a-24fd59929530 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:00.428526 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80f884-8e79-4dd7-8614-a2dd822ee37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:00.429210 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80f884-8e79-4dd7-8614-a2dd822ee37d None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:00.429440 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80f884-8e79-4dd7-8614-a2dd822ee37d None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:00.429714 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80f884-8e79-4dd7-8614-a2dd822ee37d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:00.438046 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a80f884-8e79-4dd7-8614-a2dd822ee37d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:00.439492 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a80f884-8e79-4dd7-8614-a2dd822ee37d None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:00.450848 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-9a80f884-8e79-4dd7-8614-a2dd822ee37d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 13:59:00.450848 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:59:00.450848 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:59:00.450848 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:59:00.450848 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:59:00.450848 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:59:00.450848 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:59:00.450848 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:59:00.450848 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:59:00.450848 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:59:00.450848 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:59:00.450848 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:59:00.450848 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:59:00.450848 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:59:00.450848 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:59:00.450848 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:59:00.452052 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:59:00.452052 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:59:00.452052 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:59:00.452052 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:59:00.452052 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:59:00.452052 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:59:00.452052 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:59:00.452052 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:59:00.452052 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:59:00.452052 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:59:00.452052 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:59:00.452052 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:59:00.452052 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:59:00.452052 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:59:00.452052 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:59:00.452052 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:59:00.452052 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:59:00.452052 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:59:00.453236 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:59:00.453236 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:59:00.453236 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:59:00.453236 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:59:00.453236 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:59:00.453236 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:59:00.453236 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:59:00.453236 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:59:00.453236 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 13:59:00.453236 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:59:00.453236 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 627/1226] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:59:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:59:00.459682 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee295b-ede7-4c6e-9ce1-aa2cfca6b582 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:00.520623 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:00.578311 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee295b-ede7-4c6e-9ce1-aa2cfca6b582 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:00.613616 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a759a07-3ddf-45fa-bb95-dc87f7fb0c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:00.614234 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a759a07-3ddf-45fa-bb95-dc87f7fb0c00 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:00.614517 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a759a07-3ddf-45fa-bb95-dc87f7fb0c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:00.614797 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a759a07-3ddf-45fa-bb95-dc87f7fb0c00 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:00.615589 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a759a07-3ddf-45fa-bb95-dc87f7fb0c00 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:59:00.615901 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a759a07-3ddf-45fa-bb95-dc87f7fb0c00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:00.617359 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a759a07-3ddf-45fa-bb95-dc87f7fb0c00 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:00.634917 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 600/1227] 10.222.0.48 () {56 vars in 1143 bytes} [Mon Aug 1 13:59:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:59:00.641513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80f884-8e79-4dd7-8614-a2dd822ee37d None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:00.750199 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80f884-8e79-4dd7-8614-a2dd822ee37d None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:00.783477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e09ffd-6c83-4145-a286-6d2ce98097cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:00.784047 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e09ffd-6c83-4145-a286-6d2ce98097cf None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:00.784330 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e09ffd-6c83-4145-a286-6d2ce98097cf None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:00.784640 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e09ffd-6c83-4145-a286-6d2ce98097cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:00.796252 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e09ffd-6c83-4145-a286-6d2ce98097cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:00.797808 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e09ffd-6c83-4145-a286-6d2ce98097cf None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:00.813775 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-89e09ffd-6c83-4145-a286-6d2ce98097cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:59:00.813775 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:59:00.813775 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:59:00.813775 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:59:00.813775 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:59:00.813775 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:59:00.813775 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:59:00.813775 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:59:00.813775 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:59:00.813775 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:59:00.813775 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:59:00.813775 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:59:00.813775 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:59:00.813775 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:59:00.813775 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:59:00.814757 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:59:00.814757 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:59:00.814757 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:59:00.814757 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:59:00.814757 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:59:00.814757 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:59:00.814757 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:59:00.814757 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:59:00.814757 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:59:00.814757 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:59:00.814757 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:59:00.814757 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:59:00.814757 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:59:00.814757 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:59:00.814757 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:59:00.814757 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:59:00.814757 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:59:00.815715 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:59:00.815715 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:59:00.815715 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:59:00.815715 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:59:00.815715 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 628/1228] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:59:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:59:00.822325 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a759a07-3ddf-45fa-bb95-dc87f7fb0c00 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:00.934644 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a759a07-3ddf-45fa-bb95-dc87f7fb0c00 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:00.937060 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:00.970121 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be80e751-3ae3-4f83-9585-27d2c5bcfb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:00.970705 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80e751-3ae3-4f83-9585-27d2c5bcfb56 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:00.970990 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80e751-3ae3-4f83-9585-27d2c5bcfb56 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:00.971267 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80e751-3ae3-4f83-9585-27d2c5bcfb56 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:00.971963 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be80e751-3ae3-4f83-9585-27d2c5bcfb56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:59:00.972272 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be80e751-3ae3-4f83-9585-27d2c5bcfb56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:00.973742 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be80e751-3ae3-4f83-9585-27d2c5bcfb56 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:00.985676 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 601/1229] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:59:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:59:00.991397 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e09ffd-6c83-4145-a286-6d2ce98097cf None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:01.001367 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:01.101672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e09ffd-6c83-4145-a286-6d2ce98097cf None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:01.142500 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2c11f-acdb-459b-8134-7695f1906336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:01.143160 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2c11f-acdb-459b-8134-7695f1906336 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:01.143454 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2c11f-acdb-459b-8134-7695f1906336 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:01.143728 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2c11f-acdb-459b-8134-7695f1906336 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:01.152748 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2c11f-acdb-459b-8134-7695f1906336 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:01.154190 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2c11f-acdb-459b-8134-7695f1906336 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:01.171239 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-9ad2c11f-acdb-459b-8134-7695f1906336 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:59:01.171239 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:59:01.171239 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:59:01.171239 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:59:01.171239 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:59:01.171239 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:59:01.171239 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:59:01.171239 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:59:01.171239 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:59:01.171239 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:59:01.171239 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:59:01.171239 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:59:01.171239 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:59:01.171239 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:59:01.171239 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:59:01.172331 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:59:01.172331 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:59:01.172331 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:59:01.172331 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:59:01.172331 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:59:01.172331 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:59:01.172331 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:59:01.172331 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:59:01.172331 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:59:01.172331 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:59:01.172331 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:59:01.172331 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:59:01.172331 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:59:01.172331 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:59:01.172331 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:59:01.172331 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:59:01.172331 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:59:01.173275 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:59:01.173275 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:59:01.173275 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:59:01.173275 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:59:01.173275 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:59:01.173275 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:59:01.173275 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:59:01.173275 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:59:01.173275 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 629/1230] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:59:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:59:01.178306 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be80e751-3ae3-4f83-9585-27d2c5bcfb56 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:01.293719 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be80e751-3ae3-4f83-9585-27d2c5bcfb56 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:01.349714 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c8e46-bf3a-4b62-80de-96b82f8e93bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:01.350541 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c8e46-bf3a-4b62-80de-96b82f8e93bb None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:01.350861 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c8e46-bf3a-4b62-80de-96b82f8e93bb None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:01.351153 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c8e46-bf3a-4b62-80de-96b82f8e93bb None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:01.352586 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c8e46-bf3a-4b62-80de-96b82f8e93bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:59:01.353018 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c8e46-bf3a-4b62-80de-96b82f8e93bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:01.355155 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c8e46-bf3a-4b62-80de-96b82f8e93bb None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:01.371303 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 602/1231] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 13:59:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:59:01.377485 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2c11f-acdb-459b-8134-7695f1906336 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:01.449828 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:01.490617 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2c11f-acdb-459b-8134-7695f1906336 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:01.545146 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f959ded-d0d9-4a15-a25f-d1a41aa53c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:01.546133 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f959ded-d0d9-4a15-a25f-d1a41aa53c8e None admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:01.546571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f959ded-d0d9-4a15-a25f-d1a41aa53c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:01.546941 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f959ded-d0d9-4a15-a25f-d1a41aa53c8e None admin] PATH_INFO: `/v3/projects/72ecec9fad964b1fbeaefd0519597229/users/e294fe796a1a4ac89041a532856ad627/roles/d3d8169e71454005a8cf0d7544ad05dc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:01.571390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f959ded-d0d9-4a15-a25f-d1a41aa53c8e None admin] RBAC: Authorizing `identity:create_grant(project_id=72ecec9fad964b1fbeaefd0519597229, user_id=e294fe796a1a4ac89041a532856ad627, role_id=d3d8169e71454005a8cf0d7544ad05dc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:01.573163 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f959ded-d0d9-4a15-a25f-d1a41aa53c8e None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:01.592531 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 630/1232] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 13:59:01 2022] PUT /identity/v3/projects/72ecec9fad964b1fbeaefd0519597229/users/e294fe796a1a4ac89041a532856ad627/roles/d3d8169e71454005a8cf0d7544ad05dc => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:59:03.704947 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa0197-e6be-4d06-bb83-7f01373b848a None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:03.705507 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa0197-e6be-4d06-bb83-7f01373b848a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:03.705640 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa0197-e6be-4d06-bb83-7f01373b848a None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:03.707015 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 603/1233] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:59:03.712835 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33d7e9-261b-4e5e-b0e3-ae120d81bb01 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:03.713265 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33d7e9-261b-4e5e-b0e3-ae120d81bb01 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:03.713396 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33d7e9-261b-4e5e-b0e3-ae120d81bb01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:03.733777 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-3c33d7e9-261b-4e5e-b0e3-ae120d81bb01 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:59:03.735100 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-3c33d7e9-261b-4e5e-b0e3-ae120d81bb01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:03.810112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 631/1234] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:59:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:59:03.834897 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa0197-e6be-4d06-bb83-7f01373b848a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:03.862213 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:03.945010 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa0197-e6be-4d06-bb83-7f01373b848a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:03.986903 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed290bf4-2430-47b3-805c-e66fa98fc6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:03.987506 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed290bf4-2430-47b3-805c-e66fa98fc6e7 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:03.987851 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed290bf4-2430-47b3-805c-e66fa98fc6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:03.988139 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed290bf4-2430-47b3-805c-e66fa98fc6e7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:03.999654 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed290bf4-2430-47b3-805c-e66fa98fc6e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:59:04.004796 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-ed290bf4-2430-47b3-805c-e66fa98fc6e7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:59:04.004796 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:59:04.004796 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:59:04.004796 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:59:04.004796 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:59:04.004796 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:59:04.004796 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:59:04.004796 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:59:04.004796 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:59:04.004796 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:59:04.004796 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:59:04.004796 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:59:04.004796 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:59:04.004796 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:59:04.004796 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:59:04.004796 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:59:04.005886 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:59:04.005886 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:59:04.005886 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:59:04.005886 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:59:04.005886 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:59:04.005886 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:59:04.005886 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:59:04.005886 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:59:04.005886 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:59:04.005886 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:59:04.005886 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:59:04.005886 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:59:04.005886 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:59:04.005886 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:59:04.005886 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:59:04.005886 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:59:04.006918 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:59:04.006918 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:59:04.006918 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:59:04.006918 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:59:04.006918 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:59:04.006918 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:59:04.006918 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:59:04.006918 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:59:04.006918 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:59:04.006918 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:59:04.006918 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 13:59:04.006918 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:59:04.006918 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 604/1235] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 13:59:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:59:04.013356 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c33d7e9-261b-4e5e-b0e3-ae120d81bb01 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:04.063474 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:04.121015 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c33d7e9-261b-4e5e-b0e3-ae120d81bb01 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:04.159529 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a919e91c-b23c-4d7d-a3c4-89e12db1ba31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:04.160172 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919e91c-b23c-4d7d-a3c4-89e12db1ba31 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:04.160460 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919e91c-b23c-4d7d-a3c4-89e12db1ba31 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:04.160807 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919e91c-b23c-4d7d-a3c4-89e12db1ba31 None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:04.161771 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a919e91c-b23c-4d7d-a3c4-89e12db1ba31 None admin] RBAC: Adding query filter params (name=service) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:59:04.162146 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a919e91c-b23c-4d7d-a3c4-89e12db1ba31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:04.163747 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a919e91c-b23c-4d7d-a3c4-89e12db1ba31 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:04.173893 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 632/1236] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 13:59:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:59:04.180275 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed290bf4-2430-47b3-805c-e66fa98fc6e7 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:04.271254 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:04.287782 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed290bf4-2430-47b3-805c-e66fa98fc6e7 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:04.330153 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b1164-99c6-4114-978e-52c1d32c2b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:04.330887 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b1164-99c6-4114-978e-52c1d32c2b03 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:04.331290 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b1164-99c6-4114-978e-52c1d32c2b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:04.331571 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b1164-99c6-4114-978e-52c1d32c2b03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:04.343020 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8b1164-99c6-4114-978e-52c1d32c2b03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:04.344470 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8b1164-99c6-4114-978e-52c1d32c2b03 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:04.365465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-8d8b1164-99c6-4114-978e-52c1d32c2b03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:59:04.365465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:59:04.365465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:59:04.365465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:59:04.365465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:59:04.365465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:59:04.365465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:59:04.365465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:59:04.365465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:59:04.365465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:59:04.365465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:59:04.365465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:59:04.365465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:59:04.365465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:59:04.365465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:59:04.366706 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:59:04.366706 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:59:04.366706 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:59:04.366706 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:59:04.366706 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:59:04.366706 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:59:04.366706 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:59:04.366706 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:59:04.366706 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:59:04.366706 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:59:04.366706 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:59:04.366706 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:59:04.366706 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:59:04.366706 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:59:04.366706 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:59:04.366706 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:59:04.366706 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:59:04.367891 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:59:04.367891 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:59:04.367891 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:59:04.367891 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:59:04.367891 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 605/1237] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:59:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:59:04.373848 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a919e91c-b23c-4d7d-a3c4-89e12db1ba31 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:04.484591 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a919e91c-b23c-4d7d-a3c4-89e12db1ba31 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:04.494360 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:04.522176 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15aa60b-ff5d-4cb2-a6f9-caf431b69f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:04.522787 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15aa60b-ff5d-4cb2-a6f9-caf431b69f0c None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:04.523098 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15aa60b-ff5d-4cb2-a6f9-caf431b69f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:04.523373 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15aa60b-ff5d-4cb2-a6f9-caf431b69f0c None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:04.524128 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15aa60b-ff5d-4cb2-a6f9-caf431b69f0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:59:04.524432 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15aa60b-ff5d-4cb2-a6f9-caf431b69f0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:04.525982 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15aa60b-ff5d-4cb2-a6f9-caf431b69f0c None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:04.544617 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 633/1238] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:59:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:59:04.546040 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b1164-99c6-4114-978e-52c1d32c2b03 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:04.658866 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b1164-99c6-4114-978e-52c1d32c2b03 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:04.698881 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df40fc0d-812a-476c-9816-e31370d62eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:04.699681 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40fc0d-812a-476c-9816-e31370d62eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:04.700021 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40fc0d-812a-476c-9816-e31370d62eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:04.700407 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40fc0d-812a-476c-9816-e31370d62eb1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:04.703644 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:04.709379 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df40fc0d-812a-476c-9816-e31370d62eb1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:04.711141 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df40fc0d-812a-476c-9816-e31370d62eb1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:04.724897 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-df40fc0d-812a-476c-9816-e31370d62eb1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 13:59:04.724897 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:59:04.724897 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:59:04.724897 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:59:04.724897 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:59:04.724897 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:59:04.724897 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:59:04.724897 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:59:04.724897 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:59:04.724897 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:59:04.724897 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:59:04.724897 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:59:04.724897 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:59:04.724897 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:59:04.724897 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:59:04.724897 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:59:04.726011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:59:04.726011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:59:04.726011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:59:04.726011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:59:04.726011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:59:04.726011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:59:04.726011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:59:04.726011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:59:04.726011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:59:04.726011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:59:04.726011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:59:04.726011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:59:04.726011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:59:04.726011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:59:04.726011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:59:04.726011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:59:04.726011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:59:04.726011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:59:04.727004 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:59:04.727004 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:59:04.727004 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:59:04.727004 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:59:04.727004 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:59:04.727004 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:59:04.727004 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:59:04.727004 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:59:04.727004 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 13:59:04.727004 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:59:04.727004 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 606/1239] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:59:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:59:04.733236 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15aa60b-ff5d-4cb2-a6f9-caf431b69f0c None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:04.843488 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15aa60b-ff5d-4cb2-a6f9-caf431b69f0c None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:04.880157 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8001b-68e5-4d3a-bb48-c97c387e8b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:04.880812 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8001b-68e5-4d3a-bb48-c97c387e8b05 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:04.881101 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8001b-68e5-4d3a-bb48-c97c387e8b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:04.881380 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8001b-68e5-4d3a-bb48-c97c387e8b05 None admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:04.882174 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e8001b-68e5-4d3a-bb48-c97c387e8b05 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:59:04.882505 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e8001b-68e5-4d3a-bb48-c97c387e8b05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:04.884323 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e8001b-68e5-4d3a-bb48-c97c387e8b05 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:04.903122 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 634/1240] 10.222.0.48 () {56 vars in 1143 bytes} [Mon Aug 1 13:59:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:59:04.909447 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df40fc0d-812a-476c-9816-e31370d62eb1 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:05.025447 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df40fc0d-812a-476c-9816-e31370d62eb1 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:05.068784 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ec5b7e-b1bd-4a61-b9bd-f41392e24fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:05.069396 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec5b7e-b1bd-4a61-b9bd-f41392e24fc3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:05.069678 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec5b7e-b1bd-4a61-b9bd-f41392e24fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:05.069987 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec5b7e-b1bd-4a61-b9bd-f41392e24fc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:05.137516 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:05.145002 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ec5b7e-b1bd-4a61-b9bd-f41392e24fc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:05.146468 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ec5b7e-b1bd-4a61-b9bd-f41392e24fc3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:05.267853 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 607/1241] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:59:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 13:59:05.269300 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8001b-68e5-4d3a-bb48-c97c387e8b05 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:05.298975 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:05.387396 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8001b-68e5-4d3a-bb48-c97c387e8b05 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:05.427408 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c5825-7cf0-43c6-902d-4c637c19924a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:05.428058 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c5825-7cf0-43c6-902d-4c637c19924a None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:05.428348 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c5825-7cf0-43c6-902d-4c637c19924a None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:05.428675 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c5825-7cf0-43c6-902d-4c637c19924a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:05.441486 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351c5825-7cf0-43c6-902d-4c637c19924a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:05.443423 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351c5825-7cf0-43c6-902d-4c637c19924a None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:05.459984 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-351c5825-7cf0-43c6-902d-4c637c19924a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:59:05.459984 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:59:05.459984 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:59:05.459984 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:59:05.459984 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:59:05.459984 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:59:05.459984 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:59:05.459984 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:59:05.459984 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:59:05.459984 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:59:05.459984 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:59:05.459984 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:59:05.459984 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:59:05.459984 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:59:05.459984 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:59:05.461130 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:59:05.461130 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:59:05.461130 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:59:05.461130 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:59:05.461130 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:59:05.461130 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:59:05.461130 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:59:05.461130 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:59:05.461130 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:59:05.461130 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:59:05.461130 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:59:05.461130 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:59:05.461130 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:59:05.461130 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:59:05.461130 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:59:05.461130 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:59:05.461130 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:59:05.462617 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:59:05.462617 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:59:05.462617 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:59:05.462617 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:59:05.462617 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 635/1242] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:59:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:59:05.468260 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ec5b7e-b1bd-4a61-b9bd-f41392e24fc3 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:05.574414 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:05.582786 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ec5b7e-b1bd-4a61-b9bd-f41392e24fc3 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:05.627826 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9c1a1-9095-4eea-9629-924a10afa651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:05.628869 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9c1a1-9095-4eea-9629-924a10afa651 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:05.629164 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9c1a1-9095-4eea-9629-924a10afa651 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:05.629449 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9c1a1-9095-4eea-9629-924a10afa651 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:05.630167 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9c1a1-9095-4eea-9629-924a10afa651 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:59:05.630478 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9c1a1-9095-4eea-9629-924a10afa651 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:05.631873 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9c1a1-9095-4eea-9629-924a10afa651 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:05.646691 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 608/1243] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:59:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:59:05.652278 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c5825-7cf0-43c6-902d-4c637c19924a None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:05.738329 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:05.762831 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c5825-7cf0-43c6-902d-4c637c19924a None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:05.797709 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4bc98e-8e02-4efa-a8a2-b50772514be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:05.798362 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bc98e-8e02-4efa-a8a2-b50772514be9 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:05.798651 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bc98e-8e02-4efa-a8a2-b50772514be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:05.798931 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bc98e-8e02-4efa-a8a2-b50772514be9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:05.808506 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4bc98e-8e02-4efa-a8a2-b50772514be9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:05.809968 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4bc98e-8e02-4efa-a8a2-b50772514be9 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:05.822700 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-2b4bc98e-8e02-4efa-a8a2-b50772514be9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:59:05.822700 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:59:05.822700 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:59:05.822700 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:59:05.822700 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:59:05.822700 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:59:05.822700 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:59:05.822700 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:59:05.822700 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:59:05.822700 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:59:05.822700 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:59:05.822700 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:59:05.822700 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:59:05.822700 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:59:05.822700 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:59:05.823788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:59:05.823788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:59:05.823788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:59:05.823788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:59:05.823788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:59:05.823788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:59:05.823788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:59:05.823788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:59:05.823788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:59:05.823788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:59:05.823788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:59:05.823788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:59:05.823788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:59:05.823788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:59:05.823788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:59:05.823788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:59:05.823788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:59:05.824864 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:59:05.824864 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:59:05.824864 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:59:05.824864 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:59:05.824864 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:59:05.824864 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:59:05.824864 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:59:05.824864 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:59:05.824864 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 636/1244] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:59:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:59:05.829881 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9c1a1-9095-4eea-9629-924a10afa651 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:05.946097 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9c1a1-9095-4eea-9629-924a10afa651 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:05.985963 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3275e5-36cb-4e54-a7f4-d4604a877848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:05.986835 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3275e5-36cb-4e54-a7f4-d4604a877848 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:05.987313 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3275e5-36cb-4e54-a7f4-d4604a877848 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:05.987802 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3275e5-36cb-4e54-a7f4-d4604a877848 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:05.988896 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3275e5-36cb-4e54-a7f4-d4604a877848 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:59:05.989273 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3275e5-36cb-4e54-a7f4-d4604a877848 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:05.991235 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3275e5-36cb-4e54-a7f4-d4604a877848 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:06.006169 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 609/1245] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 13:59:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:59:06.013006 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4bc98e-8e02-4efa-a8a2-b50772514be9 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:06.139087 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4bc98e-8e02-4efa-a8a2-b50772514be9 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:06.179170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ceb38-1995-4512-96c2-c13d3b4a91ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:06.179977 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ceb38-1995-4512-96c2-c13d3b4a91ad None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:06.180466 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ceb38-1995-4512-96c2-c13d3b4a91ad None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:06.180813 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ceb38-1995-4512-96c2-c13d3b4a91ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:06.181880 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5ceb38-1995-4512-96c2-c13d3b4a91ad None admin] RBAC: Adding query filter params (role.id=d3d8169e71454005a8cf0d7544ad05dc, scope.project.id=72ecec9fad964b1fbeaefd0519597229, user.id=e294fe796a1a4ac89041a532856ad627) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:59:06.182879 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5ceb38-1995-4512-96c2-c13d3b4a91ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:06.184944 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5ceb38-1995-4512-96c2-c13d3b4a91ad None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:06.191879 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 637/1246] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:59:06 2022] GET /identity/v3/role_assignments?user.id=e294fe796a1a4ac89041a532856ad627&scope.project.id=72ecec9fad964b1fbeaefd0519597229&role.id=d3d8169e71454005a8cf0d7544ad05dc => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:59:08.605618 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178408a4-2503-4a89-8efb-862ea848407e None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:08.606245 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178408a4-2503-4a89-8efb-862ea848407e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:08.606245 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178408a4-2503-4a89-8efb-862ea848407e None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:08.607495 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 610/1247] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:59:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:59:08.612670 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2809a9-31d6-49d4-a097-93604782b131 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:08.613138 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2809a9-31d6-49d4-a097-93604782b131 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:08.613296 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2809a9-31d6-49d4-a097-93604782b131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:08.620844 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:08.634749 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-2d2809a9-31d6-49d4-a097-93604782b131 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:59:08.635988 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-2d2809a9-31d6-49d4-a097-93604782b131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:08.708620 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 638/1248] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:59:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:59:08.735515 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178408a4-2503-4a89-8efb-862ea848407e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:08.757426 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:08.855554 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178408a4-2503-4a89-8efb-862ea848407e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:08.894694 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869525f-8562-4e44-8c66-cb10a6f4b6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:08.895271 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869525f-8562-4e44-8c66-cb10a6f4b6b1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:08.895557 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869525f-8562-4e44-8c66-cb10a6f4b6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:08.895836 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869525f-8562-4e44-8c66-cb10a6f4b6b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:08.907474 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8869525f-8562-4e44-8c66-cb10a6f4b6b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:59:08.912385 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-8869525f-8562-4e44-8c66-cb10a6f4b6b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:59:08.912385 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:59:08.912385 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:59:08.912385 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:59:08.912385 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:59:08.912385 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:59:08.912385 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:59:08.912385 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:59:08.912385 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:59:08.912385 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:59:08.912385 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:59:08.912385 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:59:08.912385 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:59:08.912385 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:59:08.912385 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:59:08.912385 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:59:08.913551 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:59:08.913551 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:59:08.913551 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:59:08.913551 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:59:08.913551 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:59:08.913551 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:59:08.913551 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:59:08.913551 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:59:08.913551 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:59:08.913551 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:59:08.913551 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:59:08.913551 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:59:08.913551 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:59:08.913551 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:59:08.913551 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:59:08.913551 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:59:08.914572 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:59:08.914572 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:59:08.914572 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:59:08.914572 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:59:08.914572 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:59:08.914572 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:59:08.914572 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:59:08.914572 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:59:08.914572 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:59:08.914572 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:59:08.914572 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:59:08.914572 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:59:08.914572 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 611/1249] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 13:59:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:59:08.920170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2809a9-31d6-49d4-a097-93604782b131 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:09.025198 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2809a9-31d6-49d4-a097-93604782b131 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:09.056822 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:09.062928 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c2881f-b089-42cd-aaac-aed7d2b222f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:09.063573 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c2881f-b089-42cd-aaac-aed7d2b222f4 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:09.063868 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c2881f-b089-42cd-aaac-aed7d2b222f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:09.064102 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c2881f-b089-42cd-aaac-aed7d2b222f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:09.064862 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c2881f-b089-42cd-aaac-aed7d2b222f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:59:09.065161 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c2881f-b089-42cd-aaac-aed7d2b222f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:09.066599 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c2881f-b089-42cd-aaac-aed7d2b222f4 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:09.078728 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 639/1250] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:59:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:59:09.084999 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869525f-8562-4e44-8c66-cb10a6f4b6b1 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:09.180515 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:09.208417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869525f-8562-4e44-8c66-cb10a6f4b6b1 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:09.249639 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15630745-4040-409c-9a5b-f6e913008e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:09.250457 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15630745-4040-409c-9a5b-f6e913008e0e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:09.250856 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15630745-4040-409c-9a5b-f6e913008e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:09.251176 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15630745-4040-409c-9a5b-f6e913008e0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:09.264563 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15630745-4040-409c-9a5b-f6e913008e0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:09.266340 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15630745-4040-409c-9a5b-f6e913008e0e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:09.285720 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-15630745-4040-409c-9a5b-f6e913008e0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:59:09.285720 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:59:09.285720 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:59:09.285720 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:59:09.285720 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:59:09.285720 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:59:09.285720 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:59:09.285720 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:59:09.285720 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:59:09.285720 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:59:09.285720 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:59:09.285720 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:59:09.285720 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:59:09.285720 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:59:09.285720 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:59:09.286888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:59:09.286888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:59:09.286888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:59:09.286888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:59:09.286888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:59:09.286888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:59:09.286888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:59:09.286888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:59:09.286888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:59:09.286888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:59:09.286888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:59:09.286888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:59:09.286888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:59:09.286888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:59:09.286888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:59:09.286888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:59:09.286888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:59:09.288436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:59:09.288436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:59:09.288436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:59:09.288436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:59:09.288436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 612/1251] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:59:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:59:09.294373 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c2881f-b089-42cd-aaac-aed7d2b222f4 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:09.418267 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c2881f-b089-42cd-aaac-aed7d2b222f4 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:09.461028 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc7eaab-e2ff-4eca-be89-a5f05baaf77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:09.461564 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7eaab-e2ff-4eca-be89-a5f05baaf77d None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:09.461851 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7eaab-e2ff-4eca-be89-a5f05baaf77d None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:09.462137 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7eaab-e2ff-4eca-be89-a5f05baaf77d None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:09.462842 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc7eaab-e2ff-4eca-be89-a5f05baaf77d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:59:09.463149 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc7eaab-e2ff-4eca-be89-a5f05baaf77d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:09.465141 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc7eaab-e2ff-4eca-be89-a5f05baaf77d None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:09.478401 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 640/1252] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:59:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:59:09.484416 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15630745-4040-409c-9a5b-f6e913008e0e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:09.604635 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15630745-4040-409c-9a5b-f6e913008e0e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:09.636036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:09.645201 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d507a00-67ab-44f9-9d55-a3c8ba41f03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:09.645859 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d507a00-67ab-44f9-9d55-a3c8ba41f03e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:09.646146 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d507a00-67ab-44f9-9d55-a3c8ba41f03e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:09.646453 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d507a00-67ab-44f9-9d55-a3c8ba41f03e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:09.656661 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d507a00-67ab-44f9-9d55-a3c8ba41f03e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:09.658127 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d507a00-67ab-44f9-9d55-a3c8ba41f03e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:09.671363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-2d507a00-67ab-44f9-9d55-a3c8ba41f03e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 13:59:09.671363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:59:09.671363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:59:09.671363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:59:09.671363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:59:09.671363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:59:09.671363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:59:09.671363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:59:09.671363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:59:09.671363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:59:09.671363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:59:09.671363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:59:09.671363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:59:09.671363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:59:09.671363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:59:09.671363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:59:09.672457 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:59:09.672457 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:59:09.672457 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:59:09.672457 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:59:09.672457 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:59:09.672457 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:59:09.672457 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:59:09.672457 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:59:09.672457 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:59:09.672457 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:59:09.672457 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:59:09.672457 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:59:09.672457 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:59:09.672457 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:59:09.672457 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:59:09.672457 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:59:09.672457 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:59:09.672457 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:59:09.673413 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:59:09.673413 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:59:09.673413 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:59:09.673413 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:59:09.673413 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:59:09.673413 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:59:09.673413 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:59:09.673413 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:59:09.673413 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 13:59:09.673413 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:59:09.673413 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 613/1253] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:59:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:59:09.678380 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc7eaab-e2ff-4eca-be89-a5f05baaf77d None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:09.746007 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:09.797567 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc7eaab-e2ff-4eca-be89-a5f05baaf77d None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:09.835109 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd09b871-c8ae-4847-9acb-facf5d318b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:09.835758 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09b871-c8ae-4847-9acb-facf5d318b62 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:09.836021 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09b871-c8ae-4847-9acb-facf5d318b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:09.836326 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09b871-c8ae-4847-9acb-facf5d318b62 None admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:09.837134 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd09b871-c8ae-4847-9acb-facf5d318b62 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:59:09.837444 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd09b871-c8ae-4847-9acb-facf5d318b62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:09.838988 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd09b871-c8ae-4847-9acb-facf5d318b62 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:09.858736 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 641/1254] 10.222.0.48 () {56 vars in 1143 bytes} [Mon Aug 1 13:59:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:59:09.865734 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d507a00-67ab-44f9-9d55-a3c8ba41f03e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:09.987435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d507a00-67ab-44f9-9d55-a3c8ba41f03e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:10.031404 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1250751-b234-4201-8aec-f3861aa44b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:10.031969 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1250751-b234-4201-8aec-f3861aa44b91 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:10.032253 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1250751-b234-4201-8aec-f3861aa44b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:10.032570 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1250751-b234-4201-8aec-f3861aa44b91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:10.096965 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:10.111725 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1250751-b234-4201-8aec-f3861aa44b91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:10.113213 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1250751-b234-4201-8aec-f3861aa44b91 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:10.225039 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 614/1255] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:59:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 13:59:10.231907 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd09b871-c8ae-4847-9acb-facf5d318b62 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:10.370919 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd09b871-c8ae-4847-9acb-facf5d318b62 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:10.388924 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:10.414499 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea7d9a-486f-48a1-ba28-fb2d82e990bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:10.415152 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea7d9a-486f-48a1-ba28-fb2d82e990bc None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:10.415436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea7d9a-486f-48a1-ba28-fb2d82e990bc None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:10.415717 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea7d9a-486f-48a1-ba28-fb2d82e990bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:10.426483 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea7d9a-486f-48a1-ba28-fb2d82e990bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:10.427921 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea7d9a-486f-48a1-ba28-fb2d82e990bc None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:10.444462 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-96ea7d9a-486f-48a1-ba28-fb2d82e990bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:59:10.444462 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:59:10.444462 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:59:10.444462 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:59:10.444462 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:59:10.444462 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:59:10.444462 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:59:10.444462 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:59:10.444462 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:59:10.444462 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:59:10.444462 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:59:10.444462 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:59:10.444462 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:59:10.444462 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:59:10.444462 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:59:10.445477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:59:10.445477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:59:10.445477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:59:10.445477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:59:10.445477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:59:10.445477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:59:10.445477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:59:10.445477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:59:10.445477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:59:10.445477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:59:10.445477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:59:10.445477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:59:10.445477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:59:10.445477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:59:10.445477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:59:10.445477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:59:10.445477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:59:10.446439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:59:10.446439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:59:10.446439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:59:10.446439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:59:10.446439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 642/1256] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:59:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:59:10.452301 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1250751-b234-4201-8aec-f3861aa44b91 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:10.540106 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:10.554300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1250751-b234-4201-8aec-f3861aa44b91 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:10.596580 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eea2df-84f7-445a-b5b1-e940e0510e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:10.597206 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eea2df-84f7-445a-b5b1-e940e0510e0f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:10.597494 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eea2df-84f7-445a-b5b1-e940e0510e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:10.597779 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eea2df-84f7-445a-b5b1-e940e0510e0f None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:10.598481 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eea2df-84f7-445a-b5b1-e940e0510e0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:59:10.598779 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eea2df-84f7-445a-b5b1-e940e0510e0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:10.600189 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eea2df-84f7-445a-b5b1-e940e0510e0f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:10.614060 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 615/1257] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:59:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:59:10.620559 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea7d9a-486f-48a1-ba28-fb2d82e990bc None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:10.745366 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea7d9a-486f-48a1-ba28-fb2d82e990bc None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:10.784855 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75305383-a21a-466e-b952-1ebf1f23b5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:10.785506 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75305383-a21a-466e-b952-1ebf1f23b5da None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:10.785793 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75305383-a21a-466e-b952-1ebf1f23b5da None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:10.786070 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75305383-a21a-466e-b952-1ebf1f23b5da None admin] PATH_INFO: `/v3/projects/service` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:10.793489 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75305383-a21a-466e-b952-1ebf1f23b5da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:10.795035 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75305383-a21a-466e-b952-1ebf1f23b5da None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:10.808233 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-75305383-a21a-466e-b952-1ebf1f23b5da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:59:10.808233 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:59:10.808233 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:59:10.808233 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:59:10.808233 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:59:10.808233 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:59:10.808233 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:59:10.808233 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:59:10.808233 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:59:10.808233 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:59:10.808233 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:59:10.808233 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:59:10.808233 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:59:10.808233 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:59:10.808233 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:59:10.809378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:59:10.809378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:59:10.809378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:59:10.809378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:59:10.809378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:59:10.809378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:59:10.809378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:59:10.809378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:59:10.809378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:59:10.809378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:59:10.809378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:59:10.809378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:59:10.809378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:59:10.809378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:59:10.809378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:59:10.809378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:59:10.809378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:59:10.810482 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:59:10.810482 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:59:10.810482 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:59:10.810482 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:59:10.810482 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:59:10.810482 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:59:10.810482 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:59:10.810482 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:59:10.810482 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 643/1258] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:59:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:59:10.815602 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eea2df-84f7-445a-b5b1-e940e0510e0f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:10.937016 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eea2df-84f7-445a-b5b1-e940e0510e0f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:10.974644 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49323c3-6a47-4caa-9b63-1ec8da049e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:10.975267 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49323c3-6a47-4caa-9b63-1ec8da049e79 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:10.975553 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49323c3-6a47-4caa-9b63-1ec8da049e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:10.975832 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49323c3-6a47-4caa-9b63-1ec8da049e79 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:10.976683 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49323c3-6a47-4caa-9b63-1ec8da049e79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:59:10.977103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49323c3-6a47-4caa-9b63-1ec8da049e79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:10.978858 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49323c3-6a47-4caa-9b63-1ec8da049e79 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:10.992253 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 616/1259] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 13:59:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:59:10.998801 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75305383-a21a-466e-b952-1ebf1f23b5da None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:11.026798 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:11.110477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75305383-a21a-466e-b952-1ebf1f23b5da None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:11.149016 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6de645b-42af-4a4f-9978-1e3d4bf54663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:11.149654 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de645b-42af-4a4f-9978-1e3d4bf54663 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:11.149939 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de645b-42af-4a4f-9978-1e3d4bf54663 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:11.150219 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de645b-42af-4a4f-9978-1e3d4bf54663 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:11.151676 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6de645b-42af-4a4f-9978-1e3d4bf54663 None admin] RBAC: Adding query filter params (role.id=58f91e3a13664dabb18c6227a1658fa6, scope.project.id=72ecec9fad964b1fbeaefd0519597229, user.id=e294fe796a1a4ac89041a532856ad627) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:59:11.151994 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6de645b-42af-4a4f-9978-1e3d4bf54663 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:11.153530 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6de645b-42af-4a4f-9978-1e3d4bf54663 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:11.162753 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 644/1260] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:59:10 2022] GET /identity/v3/role_assignments?user.id=e294fe796a1a4ac89041a532856ad627&scope.project.id=72ecec9fad964b1fbeaefd0519597229&role.id=58f91e3a13664dabb18c6227a1658fa6 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:59:13.146262 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0249408-96c9-4464-bad0-55ab98a27523 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:13.146756 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0249408-96c9-4464-bad0-55ab98a27523 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:13.146756 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0249408-96c9-4464-bad0-55ab98a27523 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:13.148067 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 617/1261] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:59:13.153668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4351d-c8b0-4214-8590-e61f13bdd83e None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:13.154098 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4351d-c8b0-4214-8590-e61f13bdd83e None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:13.154223 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4351d-c8b0-4214-8590-e61f13bdd83e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:13.171712 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-74c4351d-c8b0-4214-8590-e61f13bdd83e None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:59:13.172991 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-74c4351d-c8b0-4214-8590-e61f13bdd83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:13.236630 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 645/1262] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:59:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:59:13.259663 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0249408-96c9-4464-bad0-55ab98a27523 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:13.270977 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:13.381996 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0249408-96c9-4464-bad0-55ab98a27523 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:13.415667 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c32db-128f-4e29-87b3-b3b972b274ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:13.416299 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c32db-128f-4e29-87b3-b3b972b274ef None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:13.416611 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c32db-128f-4e29-87b3-b3b972b274ef None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:13.416892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c32db-128f-4e29-87b3-b3b972b274ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:13.426650 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c32db-128f-4e29-87b3-b3b972b274ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:59:13.431509 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-d73c32db-128f-4e29-87b3-b3b972b274ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:59:13.431509 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:59:13.431509 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:59:13.431509 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:59:13.431509 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:59:13.431509 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:59:13.431509 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:59:13.431509 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:59:13.431509 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:59:13.431509 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:59:13.431509 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:59:13.431509 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:59:13.431509 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:59:13.431509 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:59:13.431509 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:59:13.431509 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:59:13.433144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:59:13.433144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:59:13.433144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:59:13.433144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:59:13.433144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:59:13.433144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:59:13.433144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:59:13.433144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:59:13.433144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:59:13.433144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:59:13.433144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:59:13.433144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:59:13.433144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:59:13.433144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:59:13.433144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:59:13.433144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:59:13.433989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:59:13.433989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:59:13.433989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:59:13.433989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:59:13.433989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:59:13.433989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:59:13.433989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:59:13.433989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:59:13.433989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:59:13.433989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:59:13.433989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:59:13.433989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:59:13.433989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 618/1263] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 13:59:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:59:13.439175 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c4351d-c8b0-4214-8590-e61f13bdd83e None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:13.457366 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:13.536363 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c4351d-c8b0-4214-8590-e61f13bdd83e None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:13.570541 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb266cd-dacd-44b2-847b-7c40ba1048b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:13.571152 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb266cd-dacd-44b2-847b-7c40ba1048b0 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:13.571438 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb266cd-dacd-44b2-847b-7c40ba1048b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:13.571717 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb266cd-dacd-44b2-847b-7c40ba1048b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:13.573080 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb266cd-dacd-44b2-847b-7c40ba1048b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:59:13.573389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb266cd-dacd-44b2-847b-7c40ba1048b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:13.574749 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb266cd-dacd-44b2-847b-7c40ba1048b0 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:13.582504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 646/1264] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:59:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:59:13.588684 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c32db-128f-4e29-87b3-b3b972b274ef None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:13.653420 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:13.702592 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c32db-128f-4e29-87b3-b3b972b274ef None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:13.734658 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b95b1d7-494b-44ec-b70d-1b506703e851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:13.735300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95b1d7-494b-44ec-b70d-1b506703e851 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:13.735587 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95b1d7-494b-44ec-b70d-1b506703e851 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:13.735866 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95b1d7-494b-44ec-b70d-1b506703e851 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:13.746629 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b95b1d7-494b-44ec-b70d-1b506703e851 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:13.748353 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b95b1d7-494b-44ec-b70d-1b506703e851 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:13.762759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-3b95b1d7-494b-44ec-b70d-1b506703e851 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:59:13.762759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:59:13.762759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:59:13.762759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:59:13.762759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:59:13.762759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:59:13.762759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:59:13.762759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:59:13.762759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:59:13.762759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:59:13.762759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:59:13.762759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:59:13.762759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:59:13.762759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:59:13.762759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:59:13.763770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:59:13.763770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:59:13.763770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:59:13.763770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:59:13.763770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:59:13.763770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:59:13.763770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:59:13.763770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:59:13.763770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:59:13.763770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:59:13.763770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:59:13.763770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:59:13.763770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:59:13.763770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:59:13.763770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:59:13.763770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:59:13.763770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:59:13.765081 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:59:13.765081 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:59:13.765081 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:59:13.765081 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:59:13.765081 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 619/1265] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:59:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:59:13.769665 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb266cd-dacd-44b2-847b-7c40ba1048b0 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:13.848815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:13.867894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb266cd-dacd-44b2-847b-7c40ba1048b0 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:13.900341 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e1fd7-c88f-4961-9f93-63c092e4025b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:13.900985 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e1fd7-c88f-4961-9f93-63c092e4025b None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:13.901270 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e1fd7-c88f-4961-9f93-63c092e4025b None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:13.901547 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e1fd7-c88f-4961-9f93-63c092e4025b None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:13.902240 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860e1fd7-c88f-4961-9f93-63c092e4025b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:59:13.902545 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860e1fd7-c88f-4961-9f93-63c092e4025b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:13.903909 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860e1fd7-c88f-4961-9f93-63c092e4025b None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:13.915197 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 647/1266] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:59:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:59:13.921330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b95b1d7-494b-44ec-b70d-1b506703e851 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:14.019213 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b95b1d7-494b-44ec-b70d-1b506703e851 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:14.038567 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:14.053095 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c9821-92d9-4c85-9113-81f98ae4a10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:14.053747 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c9821-92d9-4c85-9113-81f98ae4a10b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:14.054032 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c9821-92d9-4c85-9113-81f98ae4a10b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:14.054308 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c9821-92d9-4c85-9113-81f98ae4a10b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:14.062877 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c9821-92d9-4c85-9113-81f98ae4a10b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:14.064304 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c9821-92d9-4c85-9113-81f98ae4a10b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:14.075471 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-e62c9821-92d9-4c85-9113-81f98ae4a10b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 13:59:14.075471 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:59:14.075471 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:59:14.075471 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:59:14.075471 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:59:14.075471 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:59:14.075471 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:59:14.075471 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:59:14.075471 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:59:14.075471 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:59:14.075471 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:59:14.075471 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:59:14.075471 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:59:14.075471 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:59:14.075471 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:59:14.075471 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:59:14.076623 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:59:14.076623 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:59:14.076623 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:59:14.076623 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:59:14.076623 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:59:14.076623 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:59:14.076623 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:59:14.076623 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:59:14.076623 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:59:14.076623 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:59:14.076623 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:59:14.076623 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:59:14.076623 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:59:14.076623 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:59:14.076623 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:59:14.076623 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:59:14.076623 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:59:14.076623 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:59:14.077712 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:59:14.077712 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:59:14.077712 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:59:14.077712 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:59:14.077712 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:59:14.077712 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:59:14.077712 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:59:14.077712 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:59:14.077712 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 13:59:14.077712 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:59:14.077712 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 620/1267] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:59:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:59:14.082519 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e1fd7-c88f-4961-9f93-63c092e4025b None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:14.178547 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e1fd7-c88f-4961-9f93-63c092e4025b None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:14.210927 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf7a438-e88f-4013-9a38-17944da15694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:14.211527 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7a438-e88f-4013-9a38-17944da15694 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:14.211803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7a438-e88f-4013-9a38-17944da15694 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:14.212073 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7a438-e88f-4013-9a38-17944da15694 None admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:14.213393 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf7a438-e88f-4013-9a38-17944da15694 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:59:14.213701 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf7a438-e88f-4013-9a38-17944da15694 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:14.215101 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf7a438-e88f-4013-9a38-17944da15694 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:14.223322 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:14.232062 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 648/1268] 10.222.0.48 () {56 vars in 1143 bytes} [Mon Aug 1 13:59:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:59:14.237857 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c9821-92d9-4c85-9113-81f98ae4a10b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:14.335394 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c9821-92d9-4c85-9113-81f98ae4a10b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:14.375856 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2496c64f-5543-43b8-8880-a19ebae1e0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:14.376518 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496c64f-5543-43b8-8880-a19ebae1e0b9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:14.376804 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496c64f-5543-43b8-8880-a19ebae1e0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:14.377078 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496c64f-5543-43b8-8880-a19ebae1e0b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:14.388382 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2496c64f-5543-43b8-8880-a19ebae1e0b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:14.389845 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2496c64f-5543-43b8-8880-a19ebae1e0b9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:14.404102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-2496c64f-5543-43b8-8880-a19ebae1e0b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:59:14.404102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:59:14.404102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:59:14.404102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:59:14.404102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:59:14.404102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:59:14.404102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:59:14.404102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:59:14.404102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:59:14.404102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:59:14.404102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:59:14.404102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:59:14.404102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:59:14.404102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:59:14.404102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:59:14.405163 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:59:14.405163 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:59:14.405163 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:59:14.405163 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:59:14.405163 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:59:14.405163 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:59:14.405163 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:59:14.405163 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:59:14.405163 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:59:14.405163 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:59:14.405163 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:59:14.405163 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:59:14.405163 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:59:14.405163 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:59:14.405163 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:59:14.405163 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:59:14.405163 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:59:14.406454 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:59:14.406454 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:59:14.406454 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:59:14.406454 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:59:14.406454 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 621/1269] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:59:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:59:14.410935 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf7a438-e88f-4013-9a38-17944da15694 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:14.507087 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf7a438-e88f-4013-9a38-17944da15694 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:14.539056 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ef3ba8-782b-414c-b24c-c975f2c517a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:14.539658 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef3ba8-782b-414c-b24c-c975f2c517a2 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:14.539938 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef3ba8-782b-414c-b24c-c975f2c517a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:14.540212 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef3ba8-782b-414c-b24c-c975f2c517a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:14.540927 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ef3ba8-782b-414c-b24c-c975f2c517a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:59:14.541768 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ef3ba8-782b-414c-b24c-c975f2c517a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:14.543111 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ef3ba8-782b-414c-b24c-c975f2c517a2 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:14.553774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 649/1270] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:59:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:59:14.559214 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2496c64f-5543-43b8-8880-a19ebae1e0b9 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:14.590814 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:14.657740 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2496c64f-5543-43b8-8880-a19ebae1e0b9 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:14.689532 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c1486-3a48-4dd2-b573-c88757123e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:14.690155 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c1486-3a48-4dd2-b573-c88757123e06 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:14.690439 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c1486-3a48-4dd2-b573-c88757123e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:14.690716 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c1486-3a48-4dd2-b573-c88757123e06 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:14.697671 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14c1486-3a48-4dd2-b573-c88757123e06 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:14.699120 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14c1486-3a48-4dd2-b573-c88757123e06 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:14.710114 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-b14c1486-3a48-4dd2-b573-c88757123e06 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:59:14.710114 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:59:14.710114 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:59:14.710114 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:59:14.710114 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:59:14.710114 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:59:14.710114 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:59:14.710114 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:59:14.710114 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:59:14.710114 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:59:14.710114 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:59:14.710114 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:59:14.710114 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:59:14.710114 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:59:14.710114 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:59:14.711144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:59:14.711144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:59:14.711144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:59:14.711144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:59:14.711144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:59:14.711144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:59:14.711144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:59:14.711144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:59:14.711144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:59:14.711144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:59:14.711144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:59:14.711144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:59:14.711144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:59:14.711144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:59:14.711144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:59:14.711144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:59:14.711144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:59:14.712093 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:59:14.712093 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:59:14.712093 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:59:14.712093 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:59:14.712093 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:59:14.712093 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:59:14.712093 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:59:14.712093 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:59:14.712093 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 622/1271] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:59:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:59:14.717679 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ef3ba8-782b-414c-b24c-c975f2c517a2 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:14.774357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:14.815243 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ef3ba8-782b-414c-b24c-c975f2c517a2 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:14.848036 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5caa8-d07c-4598-a813-2fbcc452be54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:14.848610 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5caa8-d07c-4598-a813-2fbcc452be54 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:14.848893 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5caa8-d07c-4598-a813-2fbcc452be54 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:14.849165 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5caa8-d07c-4598-a813-2fbcc452be54 None admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:14.849904 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c5caa8-d07c-4598-a813-2fbcc452be54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:59:14.850208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c5caa8-d07c-4598-a813-2fbcc452be54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:14.851614 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c5caa8-d07c-4598-a813-2fbcc452be54 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:14.863033 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 650/1272] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 13:59:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:59:14.869117 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c1486-3a48-4dd2-b573-c88757123e06 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:14.958358 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:14.967064 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c1486-3a48-4dd2-b573-c88757123e06 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:14.999488 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843fce4-1cdb-4538-bb3e-03e293950fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:15.000140 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843fce4-1cdb-4538-bb3e-03e293950fad None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:15.000424 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843fce4-1cdb-4538-bb3e-03e293950fad None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:15.000726 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843fce4-1cdb-4538-bb3e-03e293950fad None admin] PATH_INFO: `/v3/projects/72ecec9fad964b1fbeaefd0519597229/users/e294fe796a1a4ac89041a532856ad627/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:15.002626 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8843fce4-1cdb-4538-bb3e-03e293950fad None admin] RBAC: Authorizing `identity:create_grant(project_id=72ecec9fad964b1fbeaefd0519597229, user_id=e294fe796a1a4ac89041a532856ad627, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:15.004043 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8843fce4-1cdb-4538-bb3e-03e293950fad None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:15.016595 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 623/1273] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 13:59:14 2022] PUT /identity/v3/projects/72ecec9fad964b1fbeaefd0519597229/users/e294fe796a1a4ac89041a532856ad627/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 13:59:17.008813 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059d4de-f9e0-4763-876d-3ad201734edd None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:17.009319 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059d4de-f9e0-4763-876d-3ad201734edd None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:17.009434 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059d4de-f9e0-4763-876d-3ad201734edd None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:17.010777 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 651/1274] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:59:17.015930 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab4025-cfe5-4239-9d81-0f3aee421578 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:17.016332 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab4025-cfe5-4239-9d81-0f3aee421578 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:17.016513 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab4025-cfe5-4239-9d81-0f3aee421578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:17.034037 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a7ab4025-cfe5-4239-9d81-0f3aee421578 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:59:17.035214 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a7ab4025-cfe5-4239-9d81-0f3aee421578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:17.099337 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 624/1275] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:59:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:59:17.123366 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059d4de-f9e0-4763-876d-3ad201734edd None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:17.221420 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059d4de-f9e0-4763-876d-3ad201734edd None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:17.238226 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:17.254965 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8fb8f6-f21f-4a86-9497-7e4c4740297d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:17.255605 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8fb8f6-f21f-4a86-9497-7e4c4740297d None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:17.255891 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8fb8f6-f21f-4a86-9497-7e4c4740297d None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:17.256171 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8fb8f6-f21f-4a86-9497-7e4c4740297d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:17.266176 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8fb8f6-f21f-4a86-9497-7e4c4740297d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 13:59:17.271062 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-da8fb8f6-f21f-4a86-9497-7e4c4740297d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:59:17.271062 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:59:17.271062 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:59:17.271062 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:59:17.271062 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:59:17.271062 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:59:17.271062 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:59:17.271062 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:59:17.271062 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:59:17.271062 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:59:17.271062 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:59:17.271062 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:59:17.271062 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 13:59:17.271062 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 13:59:17.271062 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 13:59:17.271062 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 13:59:17.272125 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 13:59:17.272125 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 13:59:17.272125 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 13:59:17.272125 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 13:59:17.272125 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:59:17.272125 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:59:17.272125 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:59:17.272125 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:59:17.272125 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:59:17.272125 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:59:17.272125 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:59:17.272125 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:59:17.272125 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:59:17.272125 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:59:17.272125 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:59:17.272125 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:59:17.273144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:59:17.273144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:59:17.273144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:59:17.273144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:59:17.273144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 13:59:17.273144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 13:59:17.273144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 13:59:17.273144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 13:59:17.273144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 13:59:17.273144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 13:59:17.273144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 13:59:17.273144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:59:17.273144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 652/1276] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 13:59:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:59:17.278689 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab4025-cfe5-4239-9d81-0f3aee421578 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:17.366897 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:17.386137 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab4025-cfe5-4239-9d81-0f3aee421578 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:17.419755 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf9a1f5-5fe7-41de-8ec8-96e420e63b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:17.420372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9a1f5-5fe7-41de-8ec8-96e420e63b74 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:17.420692 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9a1f5-5fe7-41de-8ec8-96e420e63b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:17.420976 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9a1f5-5fe7-41de-8ec8-96e420e63b74 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:17.421692 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf9a1f5-5fe7-41de-8ec8-96e420e63b74 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:59:17.422001 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf9a1f5-5fe7-41de-8ec8-96e420e63b74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:17.423705 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf9a1f5-5fe7-41de-8ec8-96e420e63b74 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:17.432148 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 625/1277] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 13:59:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:59:17.438293 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8fb8f6-f21f-4a86-9497-7e4c4740297d None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:17.548536 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8fb8f6-f21f-4a86-9497-7e4c4740297d None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:17.580777 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50efc9ce-fda0-4d31-b720-abb30e460615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:17.581409 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50efc9ce-fda0-4d31-b720-abb30e460615 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:17.581691 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50efc9ce-fda0-4d31-b720-abb30e460615 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:17.581969 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50efc9ce-fda0-4d31-b720-abb30e460615 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:17.592689 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50efc9ce-fda0-4d31-b720-abb30e460615 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:17.594138 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50efc9ce-fda0-4d31-b720-abb30e460615 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:17.607933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-50efc9ce-fda0-4d31-b720-abb30e460615 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:59:17.607933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:59:17.607933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:59:17.607933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:59:17.607933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:59:17.607933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:59:17.607933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:59:17.607933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:59:17.607933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:59:17.607933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:59:17.607933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:59:17.607933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:59:17.607933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:59:17.607933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:59:17.607933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:59:17.609004 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:59:17.609004 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:59:17.609004 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:59:17.609004 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:59:17.609004 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:59:17.609004 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:59:17.609004 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:59:17.609004 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:59:17.609004 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:59:17.609004 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:59:17.609004 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:59:17.609004 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:59:17.609004 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:59:17.609004 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:59:17.609004 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:59:17.609004 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:59:17.609004 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:59:17.610243 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:59:17.610243 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:59:17.610243 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:59:17.610243 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:59:17.610243 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 653/1278] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:59:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:59:17.614863 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf9a1f5-5fe7-41de-8ec8-96e420e63b74 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:17.711207 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf9a1f5-5fe7-41de-8ec8-96e420e63b74 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:17.744694 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb753bfd-b803-4a9e-848f-7f58607b4b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:17.745316 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb753bfd-b803-4a9e-848f-7f58607b4b2f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:17.745602 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb753bfd-b803-4a9e-848f-7f58607b4b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:17.745879 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb753bfd-b803-4a9e-848f-7f58607b4b2f None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:17.746569 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb753bfd-b803-4a9e-848f-7f58607b4b2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:59:17.746877 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb753bfd-b803-4a9e-848f-7f58607b4b2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:17.748522 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb753bfd-b803-4a9e-848f-7f58607b4b2f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:17.760062 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 626/1279] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:59:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:59:17.765837 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50efc9ce-fda0-4d31-b720-abb30e460615 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:17.800986 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:17.868905 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50efc9ce-fda0-4d31-b720-abb30e460615 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:17.901195 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55760d54-3d0d-4f2e-a8a5-5457e75a8c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:17.901823 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55760d54-3d0d-4f2e-a8a5-5457e75a8c51 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:17.902103 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55760d54-3d0d-4f2e-a8a5-5457e75a8c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:17.902374 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55760d54-3d0d-4f2e-a8a5-5457e75a8c51 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:17.909495 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55760d54-3d0d-4f2e-a8a5-5457e75a8c51 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:17.910924 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55760d54-3d0d-4f2e-a8a5-5457e75a8c51 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:17.922228 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-55760d54-3d0d-4f2e-a8a5-5457e75a8c51 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 13:59:17.922228 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:59:17.922228 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:59:17.922228 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:59:17.922228 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:59:17.922228 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:59:17.922228 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:59:17.922228 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:59:17.922228 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:59:17.922228 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:59:17.922228 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:59:17.922228 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:59:17.922228 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 13:59:17.922228 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 13:59:17.922228 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 13:59:17.922228 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 13:59:17.923308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:59:17.923308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:59:17.923308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 13:59:17.923308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:59:17.923308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 13:59:17.923308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 13:59:17.923308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:59:17.923308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:59:17.923308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:59:17.923308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:59:17.923308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:59:17.923308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:59:17.923308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:59:17.923308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:59:17.923308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:59:17.923308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:59:17.923308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:59:17.923308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:59:17.924593 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:59:17.924593 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:59:17.924593 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 13:59:17.924593 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 13:59:17.924593 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 13:59:17.924593 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 13:59:17.924593 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 13:59:17.924593 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 13:59:17.924593 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 13:59:17.924593 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:59:17.924593 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 654/1280] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:59:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:59:17.929118 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb753bfd-b803-4a9e-848f-7f58607b4b2f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:17.933491 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:18.025015 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb753bfd-b803-4a9e-848f-7f58607b4b2f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:18.057739 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df20e08e-9594-4548-a624-2b9525b72cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:18.058359 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20e08e-9594-4548-a624-2b9525b72cfe None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:18.058645 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20e08e-9594-4548-a624-2b9525b72cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:18.058923 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20e08e-9594-4548-a624-2b9525b72cfe None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:18.059718 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df20e08e-9594-4548-a624-2b9525b72cfe None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:59:18.060030 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df20e08e-9594-4548-a624-2b9525b72cfe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:18.061479 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df20e08e-9594-4548-a624-2b9525b72cfe None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:18.079179 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 627/1281] 10.222.0.48 () {56 vars in 1143 bytes} [Mon Aug 1 13:59:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:59:18.084911 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55760d54-3d0d-4f2e-a8a5-5457e75a8c51 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:18.175153 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:18.185003 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55760d54-3d0d-4f2e-a8a5-5457e75a8c51 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:18.217424 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f4e73-de6b-4e65-bb7d-9eacd35a078f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:18.218039 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f4e73-de6b-4e65-bb7d-9eacd35a078f None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:18.218326 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f4e73-de6b-4e65-bb7d-9eacd35a078f None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:18.218607 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f4e73-de6b-4e65-bb7d-9eacd35a078f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:18.284190 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327f4e73-de6b-4e65-bb7d-9eacd35a078f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:18.285647 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327f4e73-de6b-4e65-bb7d-9eacd35a078f None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:18.375925 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:18.386983 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 655/1282] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 13:59:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 13:59:18.394165 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df20e08e-9594-4548-a624-2b9525b72cfe None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:18.455076 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:18.493157 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df20e08e-9594-4548-a624-2b9525b72cfe None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:18.526729 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583ae2f-bb06-4081-a811-bf4626e80c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:18.527362 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583ae2f-bb06-4081-a811-bf4626e80c03 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:18.527651 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583ae2f-bb06-4081-a811-bf4626e80c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:18.527928 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583ae2f-bb06-4081-a811-bf4626e80c03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:18.537724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7583ae2f-bb06-4081-a811-bf4626e80c03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:18.539103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7583ae2f-bb06-4081-a811-bf4626e80c03 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:18.553157 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-7583ae2f-bb06-4081-a811-bf4626e80c03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:59:18.553157 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:59:18.553157 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:59:18.553157 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:59:18.553157 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:59:18.553157 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:59:18.553157 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:59:18.553157 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:59:18.553157 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:59:18.553157 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:59:18.553157 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:59:18.553157 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:59:18.553157 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 13:59:18.553157 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 13:59:18.553157 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 13:59:18.554236 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 13:59:18.554236 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:59:18.554236 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:59:18.554236 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:59:18.554236 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:59:18.554236 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:59:18.554236 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:59:18.554236 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:59:18.554236 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:59:18.554236 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:59:18.554236 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:59:18.554236 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:59:18.554236 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:59:18.554236 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:59:18.554236 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:59:18.554236 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:59:18.554236 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:59:18.555347 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 13:59:18.555347 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 13:59:18.555347 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 13:59:18.555347 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:59:18.555347 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 628/1283] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 13:59:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:59:18.559962 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f4e73-de6b-4e65-bb7d-9eacd35a078f None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:18.655681 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f4e73-de6b-4e65-bb7d-9eacd35a078f None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:18.688441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569ed6b-e8e8-41d5-a54b-4b2f9e8d1d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:18.689078 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569ed6b-e8e8-41d5-a54b-4b2f9e8d1d94 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:18.689357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569ed6b-e8e8-41d5-a54b-4b2f9e8d1d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:18.689632 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569ed6b-e8e8-41d5-a54b-4b2f9e8d1d94 None admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:18.690321 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d569ed6b-e8e8-41d5-a54b-4b2f9e8d1d94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:59:18.690626 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d569ed6b-e8e8-41d5-a54b-4b2f9e8d1d94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:18.691979 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d569ed6b-e8e8-41d5-a54b-4b2f9e8d1d94 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:18.703307 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 656/1284] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 13:59:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:59:18.708867 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583ae2f-bb06-4081-a811-bf4626e80c03 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:18.805460 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583ae2f-bb06-4081-a811-bf4626e80c03 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:18.819287 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:18.839428 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347feff1-b73a-4c28-bec5-ca9fc459bf1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:18.840064 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347feff1-b73a-4c28-bec5-ca9fc459bf1e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:18.840345 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347feff1-b73a-4c28-bec5-ca9fc459bf1e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:18.840663 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347feff1-b73a-4c28-bec5-ca9fc459bf1e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:18.847120 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347feff1-b73a-4c28-bec5-ca9fc459bf1e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:18.848573 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347feff1-b73a-4c28-bec5-ca9fc459bf1e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:18.859553 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-347feff1-b73a-4c28-bec5-ca9fc459bf1e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:59:18.859553 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:59:18.859553 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:59:18.859553 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:59:18.859553 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:59:18.859553 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:59:18.859553 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:59:18.859553 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:59:18.859553 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:59:18.859553 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:59:18.859553 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:59:18.859553 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:59:18.859553 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 13:59:18.859553 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 13:59:18.859553 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 13:59:18.860560 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 13:59:18.860560 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:59:18.860560 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:59:18.860560 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:59:18.860560 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:59:18.860560 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:59:18.860560 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:59:18.860560 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:59:18.860560 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Aug 01 13:59:18.860560 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:59:18.860560 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:59:18.860560 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:59:18.860560 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:59:18.860560 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:59:18.860560 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:59:18.860560 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:59:18.860560 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:59:18.861451 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 13:59:18.861451 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 13:59:18.861451 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 13:59:18.861451 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 13:59:18.861451 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 13:59:18.861451 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 13:59:18.861451 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 13:59:18.861451 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Aug 01 13:59:18.861451 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 629/1285] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 13:59:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:59:18.866509 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569ed6b-e8e8-41d5-a54b-4b2f9e8d1d94 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:18.916910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:18.966168 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569ed6b-e8e8-41d5-a54b-4b2f9e8d1d94 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:19.000226 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da7d86f-e32a-4781-8199-5c2a1d8d4dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:19.000862 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7d86f-e32a-4781-8199-5c2a1d8d4dda None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:19.001151 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7d86f-e32a-4781-8199-5c2a1d8d4dda None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:19.001432 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7d86f-e32a-4781-8199-5c2a1d8d4dda None admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:19.002180 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da7d86f-e32a-4781-8199-5c2a1d8d4dda None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:59:19.002491 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da7d86f-e32a-4781-8199-5c2a1d8d4dda None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:19.003912 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da7d86f-e32a-4781-8199-5c2a1d8d4dda None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:19.016056 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 657/1286] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 13:59:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:59:19.021982 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347feff1-b73a-4c28-bec5-ca9fc459bf1e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:19.117575 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347feff1-b73a-4c28-bec5-ca9fc459bf1e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:19.150324 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5239cf5-350a-42f3-ae51-69fa60d06510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:19.150921 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5239cf5-350a-42f3-ae51-69fa60d06510 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:19.151204 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5239cf5-350a-42f3-ae51-69fa60d06510 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:19.151484 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5239cf5-350a-42f3-ae51-69fa60d06510 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:19.152261 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5239cf5-350a-42f3-ae51-69fa60d06510 None admin] RBAC: Adding query filter params (role.id=58f91e3a13664dabb18c6227a1658fa6, scope.project.id=72ecec9fad964b1fbeaefd0519597229, user.id=e294fe796a1a4ac89041a532856ad627) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:59:19.152603 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5239cf5-350a-42f3-ae51-69fa60d06510 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:19.154014 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5239cf5-350a-42f3-ae51-69fa60d06510 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:19.159586 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 630/1287] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 13:59:19 2022] GET /identity/v3/role_assignments?user.id=e294fe796a1a4ac89041a532856ad627&scope.project.id=72ecec9fad964b1fbeaefd0519597229&role.id=58f91e3a13664dabb18c6227a1658fa6 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:59:21.125082 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f15bf-c7ac-48eb-b018-3a8706a135b2 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:21.125575 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f15bf-c7ac-48eb-b018-3a8706a135b2 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:21.125575 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f15bf-c7ac-48eb-b018-3a8706a135b2 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:21.126882 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 658/1288] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:59:21.131838 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1cb6c5-2804-4f59-bb0a-4d3a2e57958e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:21.132257 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1cb6c5-2804-4f59-bb0a-4d3a2e57958e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:21.132399 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1cb6c5-2804-4f59-bb0a-4d3a2e57958e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:21.150582 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-3d1cb6c5-2804-4f59-bb0a-4d3a2e57958e None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:59:21.151767 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3d1cb6c5-2804-4f59-bb0a-4d3a2e57958e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:21.180511 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:21.219557 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 631/1289] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:59:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:59:21.241766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23f15bf-c7ac-48eb-b018-3a8706a135b2 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:21.341978 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23f15bf-c7ac-48eb-b018-3a8706a135b2 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:21.354493 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:21.385304 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e66f0d-3a1e-49bc-89b3-04ddd88e0e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:21.385882 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e66f0d-3a1e-49bc-89b3-04ddd88e0e8a None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:21.386167 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e66f0d-3a1e-49bc-89b3-04ddd88e0e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:21.386444 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e66f0d-3a1e-49bc-89b3-04ddd88e0e8a None admin] PATH_INFO: `/v3/services/placement` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:21.387081 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e66f0d-3a1e-49bc-89b3-04ddd88e0e8a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:21.388468 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e66f0d-3a1e-49bc-89b3-04ddd88e0e8a None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:21.397436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-87e66f0d-3a1e-49bc-89b3-04ddd88e0e8a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 13:59:21.397436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:59:21.397436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:59:21.397436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:59:21.397436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:59:21.397436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:59:21.397436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:59:21.397436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:59:21.397436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 13:59:21.397436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:59:21.397436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:59:21.397436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:59:21.397436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:59:21.397436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:59:21.397436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:59:21.397436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:59:21.397436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:59:21.398667 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:59:21.398667 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:59:21.398667 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:59:21.398667 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:59:21.398667 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:59:21.398667 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:59:21.398667 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:59:21.398667 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:59:21.398667 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:59:21.398667 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:59:21.398667 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:59:21.398667 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:59:21.398667 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:59:21.398667 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:59:21.398667 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:59:21.398667 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:59:21.399513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:59:21.399513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:59:21.399513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:59:21.399513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:59:21.399513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:59:21.399513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:59:21.399513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:59:21.399513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:59:21.399513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:59:21.399513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:59:21.399513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:59:21.399513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:59:21.399513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 13:59:21.399513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:59:21.399513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 659/1290] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 1 13:59:21 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:59:21.405304 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1cb6c5-2804-4f59-bb0a-4d3a2e57958e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:21.500459 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1cb6c5-2804-4f59-bb0a-4d3a2e57958e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:21.534010 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f78c4c-6408-490a-9ae2-846a536444fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:21.534627 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f78c4c-6408-490a-9ae2-846a536444fa None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:21.534913 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f78c4c-6408-490a-9ae2-846a536444fa None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:21.535193 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f78c4c-6408-490a-9ae2-846a536444fa None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:21.535876 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f78c4c-6408-490a-9ae2-846a536444fa None admin] RBAC: Adding query filter params (name=placement) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:59:21.536185 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f78c4c-6408-490a-9ae2-846a536444fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:21.537579 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f78c4c-6408-490a-9ae2-846a536444fa None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:21.542292 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 632/1291] 10.222.0.48 () {56 vars in 1116 bytes} [Mon Aug 1 13:59:21 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:59:21.548009 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e66f0d-3a1e-49bc-89b3-04ddd88e0e8a None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:21.645891 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e66f0d-3a1e-49bc-89b3-04ddd88e0e8a None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:21.687256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1f525-f0f3-45a7-abc7-04297add8c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:21.687885 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1f525-f0f3-45a7-abc7-04297add8c75 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:21.688170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1f525-f0f3-45a7-abc7-04297add8c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:21.688447 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1f525-f0f3-45a7-abc7-04297add8c75 None admin] PATH_INFO: `/v3/services` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:21.689161 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc1f525-f0f3-45a7-abc7-04297add8c75 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:59:21.689467 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc1f525-f0f3-45a7-abc7-04297add8c75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:21.690846 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc1f525-f0f3-45a7-abc7-04297add8c75 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:21.695195 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 660/1292] 10.222.0.48 () {56 vars in 1116 bytes} [Mon Aug 1 13:59:21 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:59:23.659244 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2ec5c-d00c-49e1-8210-30874a1a3fee None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:23.659800 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2ec5c-d00c-49e1-8210-30874a1a3fee None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:23.659800 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2ec5c-d00c-49e1-8210-30874a1a3fee None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:23.661180 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 633/1293] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:59:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:59:23.666821 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f3a40-653d-4390-b475-e21a8475b8bc None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:23.667236 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f3a40-653d-4390-b475-e21a8475b8bc None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:23.667385 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f3a40-653d-4390-b475-e21a8475b8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:23.685863 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-1a9f3a40-653d-4390-b475-e21a8475b8bc None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:59:23.687053 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-1a9f3a40-653d-4390-b475-e21a8475b8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:23.716270 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:23.754039 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 661/1294] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:59:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:59:23.776450 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2ec5c-d00c-49e1-8210-30874a1a3fee None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:23.808138 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:23.877993 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2ec5c-d00c-49e1-8210-30874a1a3fee None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:23.910080 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a763b0-65aa-48e3-88fc-3b0b22d5faf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:23.910635 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a763b0-65aa-48e3-88fc-3b0b22d5faf0 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:23.910922 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a763b0-65aa-48e3-88fc-3b0b22d5faf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:23.911203 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a763b0-65aa-48e3-88fc-3b0b22d5faf0 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:23.911889 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a763b0-65aa-48e3-88fc-3b0b22d5faf0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:23.913272 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a763b0-65aa-48e3-88fc-3b0b22d5faf0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:23.927798 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 634/1295] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 13:59:23 2022] POST /identity/v3/services => generated 246 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:59:25.898243 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc09dc-ca14-4cde-88f7-d50546f2acfd None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:25.898777 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc09dc-ca14-4cde-88f7-d50546f2acfd None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:25.898777 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc09dc-ca14-4cde-88f7-d50546f2acfd None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:25.900239 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 662/1296] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:59:25.905395 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385672a0-8c2b-42b8-8936-fc17256a4335 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:25.905798 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385672a0-8c2b-42b8-8936-fc17256a4335 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:25.905954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385672a0-8c2b-42b8-8936-fc17256a4335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:25.924003 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-385672a0-8c2b-42b8-8936-fc17256a4335 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:59:25.925241 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-385672a0-8c2b-42b8-8936-fc17256a4335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:25.996321 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 635/1297] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:59:25 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:59:26.019445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc09dc-ca14-4cde-88f7-d50546f2acfd None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:26.118052 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc09dc-ca14-4cde-88f7-d50546f2acfd None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:26.151804 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ce465-57cc-4d32-b48f-b794f411e260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:26.152443 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ce465-57cc-4d32-b48f-b794f411e260 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:26.152762 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ce465-57cc-4d32-b48f-b794f411e260 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:26.153041 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ce465-57cc-4d32-b48f-b794f411e260 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:26.153810 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ce465-57cc-4d32-b48f-b794f411e260 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:26.155917 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ce465-57cc-4d32-b48f-b794f411e260 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:26.169653 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-3a4ce465-57cc-4d32-b48f-b794f411e260 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 13:59:26.169653 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:59:26.169653 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:59:26.169653 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:59:26.169653 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:59:26.169653 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:59:26.169653 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:59:26.169653 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:59:26.169653 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 13:59:26.169653 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:59:26.169653 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:59:26.169653 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:59:26.169653 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:59:26.169653 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:59:26.169653 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:59:26.169653 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:59:26.169653 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:59:26.170703 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:59:26.170703 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:59:26.170703 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:59:26.170703 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:59:26.170703 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:59:26.170703 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:59:26.170703 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:59:26.170703 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:59:26.170703 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:59:26.170703 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:59:26.170703 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:59:26.170703 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:59:26.170703 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:59:26.170703 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:59:26.170703 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:59:26.170703 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:59:26.171677 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:59:26.171677 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:59:26.171677 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:59:26.171677 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:59:26.171677 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:59:26.171677 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:59:26.171677 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:59:26.171677 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:59:26.171677 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:59:26.171677 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:59:26.171677 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:59:26.171677 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:59:26.171677 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 13:59:26.171677 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:59:26.171677 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 663/1298] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 1 13:59:26 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:59:26.177299 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385672a0-8c2b-42b8-8936-fc17256a4335 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:26.201272 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:26.280887 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385672a0-8c2b-42b8-8936-fc17256a4335 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:26.313524 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4050f9c-8083-4d78-ac23-fbe722ef9974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:26.314137 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4050f9c-8083-4d78-ac23-fbe722ef9974 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:26.314422 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4050f9c-8083-4d78-ac23-fbe722ef9974 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:26.314699 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4050f9c-8083-4d78-ac23-fbe722ef9974 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:26.315376 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4050f9c-8083-4d78-ac23-fbe722ef9974 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:59:26.315681 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4050f9c-8083-4d78-ac23-fbe722ef9974 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:26.317069 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4050f9c-8083-4d78-ac23-fbe722ef9974 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:26.322068 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 636/1299] 10.222.0.48 () {56 vars in 1116 bytes} [Mon Aug 1 13:59:26 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:59:26.328670 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ce465-57cc-4d32-b48f-b794f411e260 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:26.364818 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:26.435683 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ce465-57cc-4d32-b48f-b794f411e260 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:26.470151 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f277ac-9116-4c7f-bdee-58c13ee20bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:26.471158 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f277ac-9116-4c7f-bdee-58c13ee20bdc None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:26.471452 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f277ac-9116-4c7f-bdee-58c13ee20bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:26.471733 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f277ac-9116-4c7f-bdee-58c13ee20bdc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:26.472523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f277ac-9116-4c7f-bdee-58c13ee20bdc None admin] RBAC: Adding query filter params (interface=public, service_id=21abfe43a32748ceb16a1481bfea5dfb, region_id=RegionOne) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:59:26.472844 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f277ac-9116-4c7f-bdee-58c13ee20bdc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:26.474214 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f277ac-9116-4c7f-bdee-58c13ee20bdc None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:26.479794 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 664/1300] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Aug 1 13:59:26 2022] GET /identity/v3/endpoints?service_id=21abfe43a32748ceb16a1481bfea5dfb&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 13:59:26.485595 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4050f9c-8083-4d78-ac23-fbe722ef9974 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:26.564496 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:26.582894 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4050f9c-8083-4d78-ac23-fbe722ef9974 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:26.616197 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913baec5-e545-490e-a871-517b88dccda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:26.616835 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913baec5-e545-490e-a871-517b88dccda4 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:26.617123 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913baec5-e545-490e-a871-517b88dccda4 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:26.617404 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913baec5-e545-490e-a871-517b88dccda4 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:26.618058 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913baec5-e545-490e-a871-517b88dccda4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:26.619412 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913baec5-e545-490e-a871-517b88dccda4 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:26.626263 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 637/1301] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Aug 1 13:59:26 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 13:59:28.586885 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7a2ed-1586-48be-8d08-0ba01a05061c None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:28.587435 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7a2ed-1586-48be-8d08-0ba01a05061c None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:28.587435 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7a2ed-1586-48be-8d08-0ba01a05061c None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:28.588832 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 665/1302] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 13:59:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 13:59:28.593917 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc470d3a-851f-47b8-b566-9abc8ae2d34e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:28.594300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc470d3a-851f-47b8-b566-9abc8ae2d34e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:28.594482 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc470d3a-851f-47b8-b566-9abc8ae2d34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:28.612474 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-fc470d3a-851f-47b8-b566-9abc8ae2d34e None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 13:59:28.613699 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fc470d3a-851f-47b8-b566-9abc8ae2d34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 13:59:28.678190 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 638/1303] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 13:59:28 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 13:59:28.701167 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7a2ed-1586-48be-8d08-0ba01a05061c None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:28.794996 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:28.806227 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7a2ed-1586-48be-8d08-0ba01a05061c None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:28.860684 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4801eeb-8d21-48b0-9909-ff5e26c7b470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:28.861253 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4801eeb-8d21-48b0-9909-ff5e26c7b470 None admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:28.861533 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4801eeb-8d21-48b0-9909-ff5e26c7b470 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:28.861805 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4801eeb-8d21-48b0-9909-ff5e26c7b470 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:28.862431 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4801eeb-8d21-48b0-9909-ff5e26c7b470 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:28.863783 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4801eeb-8d21-48b0-9909-ff5e26c7b470 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:28.873354 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application [None req-b4801eeb-8d21-48b0-9909-ff5e26c7b470 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 13:59:28.873354 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:59:28.873354 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 13:59:28.873354 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 13:59:28.873354 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 13:59:28.873354 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 13:59:28.873354 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 13:59:28.873354 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:59:28.873354 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 13:59:28.873354 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:59:28.873354 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 13:59:28.873354 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:59:28.873354 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 13:59:28.873354 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 13:59:28.873354 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 13:59:28.873354 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 13:59:28.873354 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 13:59:28.874511 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 13:59:28.874511 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 13:59:28.874511 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 13:59:28.874511 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 13:59:28.874511 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 13:59:28.874511 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 13:59:28.874511 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 13:59:28.874511 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 13:59:28.874511 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 13:59:28.874511 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 13:59:28.874511 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 13:59:28.874511 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 13:59:28.874511 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 13:59:28.874511 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 13:59:28.874511 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 13:59:28.874511 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 13:59:28.875723 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 13:59:28.875723 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application with Lock( Aug 01 13:59:28.875723 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 13:59:28.875723 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self._enter() Aug 01 13:59:28.875723 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 13:59:28.875723 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 13:59:28.875723 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 13:59:28.875723 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application return self.creator() Aug 01 13:59:28.875723 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 13:59:28.875723 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application created_value = creator( Aug 01 13:59:28.875723 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 13:59:28.875723 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 13:59:28.875723 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 13:59:28.875723 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: ERROR keystone.server.flask.application Aug 01 13:59:28.875723 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 666/1304] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 1 13:59:28 2022] GET /identity/v3/services/placement => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 13:59:28.880567 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc470d3a-851f-47b8-b566-9abc8ae2d34e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:28.953328 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:28.975198 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc470d3a-851f-47b8-b566-9abc8ae2d34e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:29.008272 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208938b3-6170-4230-82ce-d3432d64508c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:29.008931 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208938b3-6170-4230-82ce-d3432d64508c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:29.009214 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208938b3-6170-4230-82ce-d3432d64508c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:29.009489 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208938b3-6170-4230-82ce-d3432d64508c None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:29.010160 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208938b3-6170-4230-82ce-d3432d64508c None admin] RBAC: Adding query filter params (name=placement) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 13:59:29.010463 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208938b3-6170-4230-82ce-d3432d64508c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:29.011813 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208938b3-6170-4230-82ce-d3432d64508c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:29.016761 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 639/1305] 10.222.0.48 () {56 vars in 1116 bytes} [Mon Aug 1 13:59:28 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 13:59:29.022461 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4801eeb-8d21-48b0-9909-ff5e26c7b470 None admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 13:59:29.118486 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4801eeb-8d21-48b0-9909-ff5e26c7b470 None admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 13:59:29.150388 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b86d267-858e-407d-843e-53029ad58257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 13:59:29.150983 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86d267-858e-407d-843e-53029ad58257 None admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 13:59:29.151266 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86d267-858e-407d-843e-53029ad58257 None admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 13:59:29.151543 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86d267-858e-407d-843e-53029ad58257 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 13:59:29.152245 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b86d267-858e-407d-843e-53029ad58257 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 13:59:29.153610 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b86d267-858e-407d-843e-53029ad58257 None admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 13:59:29.173076 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 667/1306] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 13:59:29 2022] POST /identity/v3/endpoints => generated 331 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 13:59:39.013929 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=30 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 13:59:39.169148 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:31.787755 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a33efe0-b251-4b6b-9486-c5b4c5c0ff73 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:31.788887 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a33efe0-b251-4b6b-9486-c5b4c5c0ff73 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:31.789393 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a33efe0-b251-4b6b-9486-c5b4c5c0ff73 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:31.790977 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 640/1307] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 14:00:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:00:31.796144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7a8b2-6690-4e6e-9c9e-a86c04f6d796 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:31.796574 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7a8b2-6690-4e6e-9c9e-a86c04f6d796 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:31.796826 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7a8b2-6690-4e6e-9c9e-a86c04f6d796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:31.949135 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-dac7a8b2-6690-4e6e-9c9e-a86c04f6d796 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:00:31.954065 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-dac7a8b2-6690-4e6e-9c9e-a86c04f6d796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:32.139410 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 668/1308] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 14:00:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:00:32.191605 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d75ce-3282-44c8-ad75-9f6b4d0c907e None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:32.191919 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d75ce-3282-44c8-ad75-9f6b4d0c907e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:32.192202 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d75ce-3282-44c8-ad75-9f6b4d0c907e None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:32.193630 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 641/1309] 10.222.0.48 () {54 vars in 965 bytes} [Mon Aug 1 14:00:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:00:32.199824 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b320f59-61dd-4d0b-8c1d-782f4e648510 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:32.200117 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b320f59-61dd-4d0b-8c1d-782f4e648510 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:32.200399 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b320f59-61dd-4d0b-8c1d-782f4e648510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:32.281475 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-9b320f59-61dd-4d0b-8c1d-782f4e648510 None None] MFA Rules not processed for user `fb9f698e09d246bcbea855613764fbe1`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:00:32.282678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-9b320f59-61dd-4d0b-8c1d-782f4e648510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:32.310470 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:32.376221 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 669/1310] 10.222.0.48 () {58 vars in 1062 bytes} [Mon Aug 1 14:00:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:00:32.386533 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d75ce-3282-44c8-ad75-9f6b4d0c907e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:00:32.388518 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fb5d75ce-3282-44c8-ad75-9f6b4d0c907e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:32.518084 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d75ce-3282-44c8-ad75-9f6b4d0c907e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:00:32.601431 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0653a20e-f5b0-4e49-9f05-b737cdaa9332 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:00:32.602332 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0653a20e-f5b0-4e49-9f05-b737cdaa9332 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:32.602767 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0653a20e-f5b0-4e49-9f05-b737cdaa9332 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:32.603209 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0653a20e-f5b0-4e49-9f05-b737cdaa9332 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:32.605054 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0653a20e-f5b0-4e49-9f05-b737cdaa9332 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:32.672305 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:32.739242 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0653a20e-f5b0-4e49-9f05-b737cdaa9332 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:00:32.741034 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0653a20e-f5b0-4e49-9f05-b737cdaa9332 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:00:32.780642 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 642/1311] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:00:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:00:35.154170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947973fc-f6d9-41ea-967b-5c1432463ca4 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:35.154685 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947973fc-f6d9-41ea-967b-5c1432463ca4 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:35.154776 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947973fc-f6d9-41ea-967b-5c1432463ca4 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:35.156101 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 670/1312] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 14:00:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:00:35.163270 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5be65a-043d-4edc-a32b-beb3d88f1b5f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:35.164212 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5be65a-043d-4edc-a32b-beb3d88f1b5f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:35.164530 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5be65a-043d-4edc-a32b-beb3d88f1b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:35.256666 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-6f5be65a-043d-4edc-a32b-beb3d88f1b5f None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:00:35.257930 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6f5be65a-043d-4edc-a32b-beb3d88f1b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:35.362338 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 643/1313] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 14:00:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:00:35.399827 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947973fc-f6d9-41ea-967b-5c1432463ca4 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:00:35.401456 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-947973fc-f6d9-41ea-967b-5c1432463ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:35.514533 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947973fc-f6d9-41ea-967b-5c1432463ca4 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:00:35.586934 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:35.589830 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba1316-778e-429f-8e66-b6758a302e73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:00:35.590506 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba1316-778e-429f-8e66-b6758a302e73 service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:35.590841 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba1316-778e-429f-8e66-b6758a302e73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:35.591320 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba1316-778e-429f-8e66-b6758a302e73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:35.593373 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-48ba1316-778e-429f-8e66-b6758a302e73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:35.703814 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ba1316-778e-429f-8e66-b6758a302e73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:00:35.705264 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ba1316-778e-429f-8e66-b6758a302e73 service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:00:35.724013 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 671/1314] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:00:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:00:38.001522 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79fe50e-a731-4f6c-bb41-84f20ac1bb24 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:38.002078 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79fe50e-a731-4f6c-bb41-84f20ac1bb24 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:38.002177 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79fe50e-a731-4f6c-bb41-84f20ac1bb24 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:38.003984 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 644/1315] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 14:00:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:00:38.009275 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f642a4e-546d-4886-b580-0ce04d375619 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:38.009678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f642a4e-546d-4886-b580-0ce04d375619 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:38.009832 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f642a4e-546d-4886-b580-0ce04d375619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:38.073183 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-1f642a4e-546d-4886-b580-0ce04d375619 None None] MFA Rules not processed for user `b3e958a355f0429dab88a0bfd6dd81e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:00:38.074363 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-1f642a4e-546d-4886-b580-0ce04d375619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:38.124458 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:38.169076 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 672/1316] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 14:00:38 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:00:38.208875 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79fe50e-a731-4f6c-bb41-84f20ac1bb24 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:00:38.210395 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f79fe50e-a731-4f6c-bb41-84f20ac1bb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:38.220089 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:38.312142 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79fe50e-a731-4f6c-bb41-84f20ac1bb24 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:00:38.388275 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d250d0-701d-4b00-9501-3c3c005db044 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:00:38.388919 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d250d0-701d-4b00-9501-3c3c005db044 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:38.389200 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d250d0-701d-4b00-9501-3c3c005db044 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:38.389473 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d250d0-701d-4b00-9501-3c3c005db044 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:38.391081 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-07d250d0-701d-4b00-9501-3c3c005db044 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:38.451634 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:38.498395 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d250d0-701d-4b00-9501-3c3c005db044 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:00:38.499819 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d250d0-701d-4b00-9501-3c3c005db044 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:00:38.517962 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 645/1317] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:00:38 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:00:41.051369 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024b318-419c-4242-9511-108b4dbd865a None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:41.051891 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024b318-419c-4242-9511-108b4dbd865a None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:41.052005 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024b318-419c-4242-9511-108b4dbd865a None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:41.053326 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 673/1318] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 14:00:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:00:41.058437 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ce284-d9e9-4d91-87a0-388fd324e8df None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:41.058882 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ce284-d9e9-4d91-87a0-388fd324e8df None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:41.059008 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ce284-d9e9-4d91-87a0-388fd324e8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:41.123512 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-1b4ce284-d9e9-4d91-87a0-388fd324e8df None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:00:41.124683 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1b4ce284-d9e9-4d91-87a0-388fd324e8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:41.222182 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 646/1319] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 14:00:41 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:00:41.260422 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024b318-419c-4242-9511-108b4dbd865a None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:00:41.261962 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-4024b318-419c-4242-9511-108b4dbd865a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:41.365950 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024b318-419c-4242-9511-108b4dbd865a None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:00:41.437355 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d2cf8-31e1-42ab-8e6a-3a04b9272a06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:00:41.437971 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d2cf8-31e1-42ab-8e6a-3a04b9272a06 service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:41.438256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d2cf8-31e1-42ab-8e6a-3a04b9272a06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:41.438533 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d2cf8-31e1-42ab-8e6a-3a04b9272a06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:41.440175 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-829d2cf8-31e1-42ab-8e6a-3a04b9272a06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:41.447439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:41.547528 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829d2cf8-31e1-42ab-8e6a-3a04b9272a06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:00:41.549007 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829d2cf8-31e1-42ab-8e6a-3a04b9272a06 service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:00:41.567193 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 674/1320] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:00:41 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:00:44.195229 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c78eb-d522-425e-ab0b-bea5e6a7b059 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:44.195767 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c78eb-d522-425e-ab0b-bea5e6a7b059 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:44.195767 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c78eb-d522-425e-ab0b-bea5e6a7b059 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:44.197179 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 647/1321] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 14:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:00:44.202500 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9633a7d-fca0-4283-9dc6-b47138fa4166 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:44.202921 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9633a7d-fca0-4283-9dc6-b47138fa4166 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:44.203185 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9633a7d-fca0-4283-9dc6-b47138fa4166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:44.271054 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-d9633a7d-fca0-4283-9dc6-b47138fa4166 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:00:44.272287 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-d9633a7d-fca0-4283-9dc6-b47138fa4166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:44.331314 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:44.372989 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 675/1322] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 14:00:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:00:44.423150 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e29884-126e-4772-8624-18bfd2ce0798 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:44.423534 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e29884-126e-4772-8624-18bfd2ce0798 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:44.423891 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e29884-126e-4772-8624-18bfd2ce0798 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:44.425398 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 648/1323] 10.222.0.48 () {54 vars in 965 bytes} [Mon Aug 1 14:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:00:44.431745 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a7ec4-d27a-44f8-9ca7-fd86869d1d9e None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:44.432039 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a7ec4-d27a-44f8-9ca7-fd86869d1d9e None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:44.432320 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a7ec4-d27a-44f8-9ca7-fd86869d1d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:44.514403 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-f91a7ec4-d27a-44f8-9ca7-fd86869d1d9e None None] MFA Rules not processed for user `fb9f698e09d246bcbea855613764fbe1`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:00:44.515553 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-f91a7ec4-d27a-44f8-9ca7-fd86869d1d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:44.595601 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 676/1324] 10.222.0.48 () {58 vars in 1062 bytes} [Mon Aug 1 14:00:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:00:44.604747 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e29884-126e-4772-8624-18bfd2ce0798 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:00:44.606290 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f0e29884-126e-4772-8624-18bfd2ce0798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:44.625596 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:44.711999 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e29884-126e-4772-8624-18bfd2ce0798 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:00:44.785942 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c5b77-cf0b-43ac-84d9-b15fdabc4459 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:00:44.786455 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c5b77-cf0b-43ac-84d9-b15fdabc4459 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:44.786740 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c5b77-cf0b-43ac-84d9-b15fdabc4459 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:44.787019 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c5b77-cf0b-43ac-84d9-b15fdabc4459 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:44.788676 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d51c5b77-cf0b-43ac-84d9-b15fdabc4459 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:44.861003 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:44.900581 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c5b77-cf0b-43ac-84d9-b15fdabc4459 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:00:44.902071 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c5b77-cf0b-43ac-84d9-b15fdabc4459 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:00:44.919941 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 649/1325] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:00:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:00:47.208826 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87312e00-b4fa-4c0b-8299-ebe0a188a4de None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:47.209349 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87312e00-b4fa-4c0b-8299-ebe0a188a4de None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:47.209447 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87312e00-b4fa-4c0b-8299-ebe0a188a4de None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:47.210752 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 677/1326] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 14:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:00:47.216117 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7fb61-8b2c-4ed2-a1be-336299b263a7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:47.216551 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7fb61-8b2c-4ed2-a1be-336299b263a7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:47.216734 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7fb61-8b2c-4ed2-a1be-336299b263a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:47.305984 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b8d7fb61-8b2c-4ed2-a1be-336299b263a7 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:00:47.307312 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b8d7fb61-8b2c-4ed2-a1be-336299b263a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:47.409993 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 650/1327] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 14:00:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:00:47.447518 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87312e00-b4fa-4c0b-8299-ebe0a188a4de None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:00:47.449196 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-87312e00-b4fa-4c0b-8299-ebe0a188a4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:47.517013 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:47.561778 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87312e00-b4fa-4c0b-8299-ebe0a188a4de None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:00:47.635053 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f172e5-2d33-42fc-8f71-d8671afd6e5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:00:47.635677 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f172e5-2d33-42fc-8f71-d8671afd6e5d service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:47.635960 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f172e5-2d33-42fc-8f71-d8671afd6e5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:47.636238 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f172e5-2d33-42fc-8f71-d8671afd6e5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:47.637922 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-97f172e5-2d33-42fc-8f71-d8671afd6e5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:47.745897 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:47.748060 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f172e5-2d33-42fc-8f71-d8671afd6e5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:00:47.749635 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f172e5-2d33-42fc-8f71-d8671afd6e5d service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:00:47.767826 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 678/1328] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:00:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:00:49.833278 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a3688-814d-49d3-b69e-75a3b3d823a7 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:49.833765 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a3688-814d-49d3-b69e-75a3b3d823a7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:49.833860 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a3688-814d-49d3-b69e-75a3b3d823a7 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:49.835215 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 651/1329] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 14:00:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:00:49.840678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a2033-f00e-400c-bddb-072a43502f51 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:49.841128 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a2033-f00e-400c-bddb-072a43502f51 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:49.841238 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a2033-f00e-400c-bddb-072a43502f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:49.906029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-f34a2033-f00e-400c-bddb-072a43502f51 None None] MFA Rules not processed for user `b3e958a355f0429dab88a0bfd6dd81e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:00:49.907139 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-f34a2033-f00e-400c-bddb-072a43502f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:50.006294 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 679/1330] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 14:00:49 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:00:50.045078 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57a3688-814d-49d3-b69e-75a3b3d823a7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:00:50.046706 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c57a3688-814d-49d3-b69e-75a3b3d823a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:50.093082 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:50.193825 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57a3688-814d-49d3-b69e-75a3b3d823a7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:00:50.273230 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef92eec0-1fa6-4a0c-88ed-aae0b6907f20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:00:50.273862 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef92eec0-1fa6-4a0c-88ed-aae0b6907f20 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:50.274149 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef92eec0-1fa6-4a0c-88ed-aae0b6907f20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:50.274430 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef92eec0-1fa6-4a0c-88ed-aae0b6907f20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:50.276056 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ef92eec0-1fa6-4a0c-88ed-aae0b6907f20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:50.354608 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:50.387967 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef92eec0-1fa6-4a0c-88ed-aae0b6907f20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:00:50.389358 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef92eec0-1fa6-4a0c-88ed-aae0b6907f20 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:00:50.408080 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 652/1331] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:00:50 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:00:52.808807 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768b993-ca69-454c-b4a3-13cb109568db None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:52.809333 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768b993-ca69-454c-b4a3-13cb109568db None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:52.809431 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768b993-ca69-454c-b4a3-13cb109568db None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:52.810710 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 680/1332] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 14:00:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:00:52.815979 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5cd87a-9ef6-40cd-bd3f-a5ae5c6986bf None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:52.816396 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5cd87a-9ef6-40cd-bd3f-a5ae5c6986bf None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:52.816579 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5cd87a-9ef6-40cd-bd3f-a5ae5c6986bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:52.881823 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-cf5cd87a-9ef6-40cd-bd3f-a5ae5c6986bf None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:00:52.883044 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cf5cd87a-9ef6-40cd-bd3f-a5ae5c6986bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:52.978050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 653/1333] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 14:00:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:00:53.015009 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b768b993-ca69-454c-b4a3-13cb109568db None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:00:53.016555 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-b768b993-ca69-454c-b4a3-13cb109568db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:53.084212 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:53.118561 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b768b993-ca69-454c-b4a3-13cb109568db None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:00:53.189497 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2b6ad-c2a0-4743-828e-64756d81e537 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:00:53.190115 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2b6ad-c2a0-4743-828e-64756d81e537 service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:53.190401 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2b6ad-c2a0-4743-828e-64756d81e537 service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:53.190685 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2b6ad-c2a0-4743-828e-64756d81e537 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:53.192282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-89e2b6ad-c2a0-4743-828e-64756d81e537 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:53.299159 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e2b6ad-c2a0-4743-828e-64756d81e537 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:00:53.300609 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e2b6ad-c2a0-4743-828e-64756d81e537 service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:00:53.310333 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:53.319474 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 681/1334] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:00:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:00:55.393425 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96aa831-b99b-40d7-8d41-788a55cf4705 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:55.394018 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96aa831-b99b-40d7-8d41-788a55cf4705 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:55.394171 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96aa831-b99b-40d7-8d41-788a55cf4705 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:55.395569 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 654/1335] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 14:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:00:55.401089 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29523dd2-c6be-476f-b3cd-dda1ec270914 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:55.401561 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29523dd2-c6be-476f-b3cd-dda1ec270914 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:55.401782 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29523dd2-c6be-476f-b3cd-dda1ec270914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:55.470818 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-29523dd2-c6be-476f-b3cd-dda1ec270914 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:00:55.472181 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-29523dd2-c6be-476f-b3cd-dda1ec270914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:55.569250 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 682/1336] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 14:00:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:00:55.609894 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96aa831-b99b-40d7-8d41-788a55cf4705 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:00:55.611452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c96aa831-b99b-40d7-8d41-788a55cf4705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:55.651378 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:55.713841 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96aa831-b99b-40d7-8d41-788a55cf4705 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:00:55.785684 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49ef58e-9bba-41a1-b3b2-8a8674ad180c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:00:55.786303 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ef58e-9bba-41a1-b3b2-8a8674ad180c service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:55.786582 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ef58e-9bba-41a1-b3b2-8a8674ad180c service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:55.786855 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ef58e-9bba-41a1-b3b2-8a8674ad180c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:55.789665 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e49ef58e-9bba-41a1-b3b2-8a8674ad180c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:55.877118 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:55.903347 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49ef58e-9bba-41a1-b3b2-8a8674ad180c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:00:55.904771 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49ef58e-9bba-41a1-b3b2-8a8674ad180c service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:00:55.923724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 655/1337] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:00:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:00:58.404932 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9420cc1-46ae-4c6f-b43b-9307daf4ec9c None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:58.405484 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9420cc1-46ae-4c6f-b43b-9307daf4ec9c None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:58.405484 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9420cc1-46ae-4c6f-b43b-9307daf4ec9c None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:58.406826 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 683/1338] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 14:00:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:00:58.411982 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7e7a3-a815-4904-adc8-cd0eff589805 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:58.412357 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7e7a3-a815-4904-adc8-cd0eff589805 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:58.412572 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7e7a3-a815-4904-adc8-cd0eff589805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:58.479140 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-eae7e7a3-a815-4904-adc8-cd0eff589805 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:00:58.480380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-eae7e7a3-a815-4904-adc8-cd0eff589805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:58.575994 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 656/1339] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 14:00:58 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:00:58.612228 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9420cc1-46ae-4c6f-b43b-9307daf4ec9c None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:00:58.613775 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-e9420cc1-46ae-4c6f-b43b-9307daf4ec9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:58.688033 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:00:58.716490 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9420cc1-46ae-4c6f-b43b-9307daf4ec9c None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:00:58.789106 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf1d681-cef2-45b8-88ab-cb524f181014 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:00:58.789724 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1d681-cef2-45b8-88ab-cb524f181014 service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:00:58.790008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1d681-cef2-45b8-88ab-cb524f181014 service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:00:58.790301 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1d681-cef2-45b8-88ab-cb524f181014 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:00:58.791910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-0bf1d681-cef2-45b8-88ab-cb524f181014 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:00:58.899409 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf1d681-cef2-45b8-88ab-cb524f181014 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:00:58.900881 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf1d681-cef2-45b8-88ab-cb524f181014 service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:00:58.918987 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 684/1340] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:00:58 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:01:02.949305 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841fb4d4-db98-499a-86eb-d570b02958f2 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:02.949820 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841fb4d4-db98-499a-86eb-d570b02958f2 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:02.949820 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841fb4d4-db98-499a-86eb-d570b02958f2 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:02.951156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 657/1341] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 14:01:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:02.956366 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c385bf9-e309-4eb8-82c1-d3ba752a4ef0 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:02.956806 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c385bf9-e309-4eb8-82c1-d3ba752a4ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:02.957105 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c385bf9-e309-4eb8-82c1-d3ba752a4ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:02.962661 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:03.026332 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-1c385bf9-e309-4eb8-82c1-d3ba752a4ef0 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:03.027461 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-1c385bf9-e309-4eb8-82c1-d3ba752a4ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:03.129522 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 685/1342] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 14:01:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:03.167951 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841fb4d4-db98-499a-86eb-d570b02958f2 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:01:03.169943 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-841fb4d4-db98-499a-86eb-d570b02958f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:03.222428 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:03.278570 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841fb4d4-db98-499a-86eb-d570b02958f2 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:01:03.358468 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc84892-49cb-4776-b5d8-c53c1682c631 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:01:03.359093 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc84892-49cb-4776-b5d8-c53c1682c631 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:03.359373 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc84892-49cb-4776-b5d8-c53c1682c631 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:03.359650 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc84892-49cb-4776-b5d8-c53c1682c631 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:03.361947 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4bc84892-49cb-4776-b5d8-c53c1682c631 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:03.457026 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:03.474932 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc84892-49cb-4776-b5d8-c53c1682c631 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:01:03.476367 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc84892-49cb-4776-b5d8-c53c1682c631 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:01:03.496237 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 658/1343] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:01:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:01:06.158452 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9964f3-2f41-44f0-8353-0ab72cf71e42 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:06.158997 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9964f3-2f41-44f0-8353-0ab72cf71e42 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:06.158997 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9964f3-2f41-44f0-8353-0ab72cf71e42 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:06.160369 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 686/1344] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 14:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:06.165747 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425b3c5-7706-4170-a531-9802124d9c23 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:06.166155 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425b3c5-7706-4170-a531-9802124d9c23 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:06.166328 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425b3c5-7706-4170-a531-9802124d9c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:06.237719 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8425b3c5-7706-4170-a531-9802124d9c23 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:06.238950 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8425b3c5-7706-4170-a531-9802124d9c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:06.336969 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 659/1345] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 14:01:06 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:06.378046 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9964f3-2f41-44f0-8353-0ab72cf71e42 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:01:06.379759 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-de9964f3-2f41-44f0-8353-0ab72cf71e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:06.466504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:06.483611 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9964f3-2f41-44f0-8353-0ab72cf71e42 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:01:06.555807 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a0ad78-1ad6-4db0-bf30-1c616b7a0ff3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:01:06.556425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a0ad78-1ad6-4db0-bf30-1c616b7a0ff3 service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:06.556737 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a0ad78-1ad6-4db0-bf30-1c616b7a0ff3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:06.557014 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a0ad78-1ad6-4db0-bf30-1c616b7a0ff3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:06.558613 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-30a0ad78-1ad6-4db0-bf30-1c616b7a0ff3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:06.666471 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a0ad78-1ad6-4db0-bf30-1c616b7a0ff3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:01:06.667924 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a0ad78-1ad6-4db0-bf30-1c616b7a0ff3 service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:01:06.686297 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 687/1346] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:01:06 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:01:10.888626 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426ae7e-f380-4163-b94d-ffea416f961f None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:10.889157 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426ae7e-f380-4163-b94d-ffea416f961f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:10.889157 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426ae7e-f380-4163-b94d-ffea416f961f None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:10.891101 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 660/1347] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 14:01:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:10.896449 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e82562-2673-41bb-b2fd-1fc5ae152bbd None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:10.896908 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e82562-2673-41bb-b2fd-1fc5ae152bbd None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:10.897071 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e82562-2673-41bb-b2fd-1fc5ae152bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:10.913260 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:10.970949 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-24e82562-2673-41bb-b2fd-1fc5ae152bbd None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:10.972170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-24e82562-2673-41bb-b2fd-1fc5ae152bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:11.072239 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 688/1348] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 14:01:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:11.110816 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6426ae7e-f380-4163-b94d-ffea416f961f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:01:11.112355 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6426ae7e-f380-4163-b94d-ffea416f961f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:11.174236 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:11.222808 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6426ae7e-f380-4163-b94d-ffea416f961f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:01:11.301493 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec22752-ac5d-447a-b1c9-731386c7f50e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:01:11.302125 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec22752-ac5d-447a-b1c9-731386c7f50e service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:11.302411 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec22752-ac5d-447a-b1c9-731386c7f50e service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:11.302691 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec22752-ac5d-447a-b1c9-731386c7f50e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:11.304324 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1ec22752-ac5d-447a-b1c9-731386c7f50e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:11.419061 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:11.423690 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec22752-ac5d-447a-b1c9-731386c7f50e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:01:11.425570 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec22752-ac5d-447a-b1c9-731386c7f50e service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:01:11.445437 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 661/1349] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:01:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:01:16.323282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52213403-11cb-4c01-9dbf-213cd5b535bc None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:16.323727 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52213403-11cb-4c01-9dbf-213cd5b535bc None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:16.323843 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52213403-11cb-4c01-9dbf-213cd5b535bc None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:16.325528 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 689/1350] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 14:01:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:16.331446 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156613d6-3e67-490d-90db-c73bff80f433 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:16.331867 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156613d6-3e67-490d-90db-c73bff80f433 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:16.337441 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156613d6-3e67-490d-90db-c73bff80f433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:16.486092 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-156613d6-3e67-490d-90db-c73bff80f433 None None] MFA Rules not processed for user `e294fe796a1a4ac89041a532856ad627`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:16.487300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-156613d6-3e67-490d-90db-c73bff80f433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:16.600412 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 662/1351] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 14:01:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:20.741117 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c7a5b-279e-41f1-9f6c-5b3e693d9bf4 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:20.742253 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c7a5b-279e-41f1-9f6c-5b3e693d9bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:20.742525 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c7a5b-279e-41f1-9f6c-5b3e693d9bf4 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:20.744390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 690/1352] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 14:01:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:20.749548 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcbe3e-483c-47e4-b155-7a919f199d91 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:20.749957 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcbe3e-483c-47e4-b155-7a919f199d91 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:20.750182 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcbe3e-483c-47e4-b155-7a919f199d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:20.809242 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:20.837022 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-3abcbe3e-483c-47e4-b155-7a919f199d91 None None] MFA Rules not processed for user `e294fe796a1a4ac89041a532856ad627`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:20.838342 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3abcbe3e-483c-47e4-b155-7a919f199d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:20.952010 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 663/1353] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 14:01:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:21.070505 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=59 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:22.320700 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b181b-5d41-46d3-9e8b-ccdd655ff116 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:22.321320 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b181b-5d41-46d3-9e8b-ccdd655ff116 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:22.321320 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b181b-5d41-46d3-9e8b-ccdd655ff116 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:22.322601 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 691/1354] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 14:01:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:22.328880 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b839bc-b10f-494c-81d1-dbc624486fce None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:22.329325 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b839bc-b10f-494c-81d1-dbc624486fce None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:22.329419 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b839bc-b10f-494c-81d1-dbc624486fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:22.405730 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b6b839bc-b10f-494c-81d1-dbc624486fce None None] MFA Rules not processed for user `e294fe796a1a4ac89041a532856ad627`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:22.406921 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b6b839bc-b10f-494c-81d1-dbc624486fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:22.449122 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:22.513800 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 664/1355] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 14:01:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:25.922917 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed8ea1-9f3e-46c3-93cc-d746ec572c97 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:25.923490 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed8ea1-9f3e-46c3-93cc-d746ec572c97 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:25.923490 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed8ea1-9f3e-46c3-93cc-d746ec572c97 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:25.925243 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 692/1356] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 14:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:25.930710 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d387e32-9d16-4091-82ea-909fac5ac867 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:25.931173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d387e32-9d16-4091-82ea-909fac5ac867 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:25.931277 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d387e32-9d16-4091-82ea-909fac5ac867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:26.022124 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-6d387e32-9d16-4091-82ea-909fac5ac867 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:26.023630 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6d387e32-9d16-4091-82ea-909fac5ac867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:26.135494 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 665/1357] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 14:01:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:26.190367 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae0451-f331-499f-a069-4eaf8c8f1214 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:26.190934 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae0451-f331-499f-a069-4eaf8c8f1214 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:26.191588 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae0451-f331-499f-a069-4eaf8c8f1214 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:26.193298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 693/1358] 10.222.0.48 () {54 vars in 962 bytes} [Mon Aug 1 14:01:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:26.200975 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ed36d-44f6-4af4-ad4c-8f461756ef76 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:26.201754 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ed36d-44f6-4af4-ad4c-8f461756ef76 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:26.202369 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ed36d-44f6-4af4-ad4c-8f461756ef76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:26.223024 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:26.293396 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-310ed36d-44f6-4af4-ad4c-8f461756ef76 None None] MFA Rules not processed for user `a246654c783b4cc68d212e91bb05d8c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:26.294588 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-310ed36d-44f6-4af4-ad4c-8f461756ef76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:26.420558 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 666/1359] 10.222.0.48 () {58 vars in 1059 bytes} [Mon Aug 1 14:01:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:26.428244 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ae0451-f331-499f-a069-4eaf8c8f1214 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:01:26.429780 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-57ae0451-f331-499f-a069-4eaf8c8f1214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:26.589964 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ae0451-f331-499f-a069-4eaf8c8f1214 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:01:26.670383 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:26.713542 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935890be-eedb-425d-ada2-420d1df3f35a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:01:26.714084 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935890be-eedb-425d-ada2-420d1df3f35a service nova] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:26.714373 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935890be-eedb-425d-ada2-420d1df3f35a service nova] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:26.714644 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935890be-eedb-425d-ada2-420d1df3f35a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:26.716248 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-935890be-eedb-425d-ada2-420d1df3f35a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:26.821794 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935890be-eedb-425d-ada2-420d1df3f35a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:01:26.823554 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935890be-eedb-425d-ada2-420d1df3f35a service nova] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:01:26.843619 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 694/1360] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:01:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:01:29.049003 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ca7f7-f0a7-4e5f-933f-2875d291045c None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:29.049480 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ca7f7-f0a7-4e5f-933f-2875d291045c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:29.049585 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ca7f7-f0a7-4e5f-933f-2875d291045c None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:29.051071 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 667/1361] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 14:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:29.056246 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d4b93-d7c9-489d-bb13-e2dbde9cdf64 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:29.056759 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d4b93-d7c9-489d-bb13-e2dbde9cdf64 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:29.056870 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d4b93-d7c9-489d-bb13-e2dbde9cdf64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:29.126296 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-072d4b93-d7c9-489d-bb13-e2dbde9cdf64 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:29.127495 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-072d4b93-d7c9-489d-bb13-e2dbde9cdf64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:29.167554 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:29.235365 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 695/1362] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 14:01:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:29.280211 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ca7f7-f0a7-4e5f-933f-2875d291045c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:01:29.281862 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-002ca7f7-f0a7-4e5f-933f-2875d291045c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:29.360643 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:29.455476 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ca7f7-f0a7-4e5f-933f-2875d291045c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:01:29.587972 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0fc75-6a9c-45b1-98bf-e487097dd6d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:01:29.588622 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0fc75-6a9c-45b1-98bf-e487097dd6d5 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:29.588905 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0fc75-6a9c-45b1-98bf-e487097dd6d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:29.589180 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0fc75-6a9c-45b1-98bf-e487097dd6d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:29.590782 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3eb0fc75-6a9c-45b1-98bf-e487097dd6d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:29.615485 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:29.690774 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb0fc75-6a9c-45b1-98bf-e487097dd6d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:01:29.692740 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb0fc75-6a9c-45b1-98bf-e487097dd6d5 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:01:29.711383 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 668/1363] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:01:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:01:32.067305 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f678c-3450-4163-abf7-0bc74b019bec None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:32.067858 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f678c-3450-4163-abf7-0bc74b019bec None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:32.067858 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f678c-3450-4163-abf7-0bc74b019bec None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:32.069451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 696/1364] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 14:01:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:32.075016 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012b8e5-6f10-4b09-a2d3-d1a79d55fd69 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:32.075838 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012b8e5-6f10-4b09-a2d3-d1a79d55fd69 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:32.076147 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012b8e5-6f10-4b09-a2d3-d1a79d55fd69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:32.166957 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f012b8e5-6f10-4b09-a2d3-d1a79d55fd69 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:32.168207 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f012b8e5-6f10-4b09-a2d3-d1a79d55fd69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:32.272225 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:32.275255 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 669/1365] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 14:01:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:32.321743 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a73e6-a43c-4605-a9b6-f9a8a061e824 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:32.322170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a73e6-a43c-4605-a9b6-f9a8a061e824 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:32.322516 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a73e6-a43c-4605-a9b6-f9a8a061e824 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:32.323921 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 697/1366] 10.222.0.48 () {54 vars in 962 bytes} [Mon Aug 1 14:01:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:32.329966 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3dd747-a36b-4317-85e5-9c7bf51d7e52 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:32.330263 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3dd747-a36b-4317-85e5-9c7bf51d7e52 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:32.330546 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3dd747-a36b-4317-85e5-9c7bf51d7e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:32.415962 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5b3dd747-a36b-4317-85e5-9c7bf51d7e52 None None] MFA Rules not processed for user `a246654c783b4cc68d212e91bb05d8c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:32.417324 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5b3dd747-a36b-4317-85e5-9c7bf51d7e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:32.547634 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 670/1367] 10.222.0.48 () {58 vars in 1059 bytes} [Mon Aug 1 14:01:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:32.555414 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a73e6-a43c-4605-a9b6-f9a8a061e824 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:01:32.556950 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-f55a73e6-a43c-4605-a9b6-f9a8a061e824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:32.715974 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a73e6-a43c-4605-a9b6-f9a8a061e824 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:01:32.737172 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:32.858588 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3dec5a-c686-4248-9b8d-c2da3e885458 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:01:32.859079 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3dec5a-c686-4248-9b8d-c2da3e885458 service nova] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:32.859369 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3dec5a-c686-4248-9b8d-c2da3e885458 service nova] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:32.859651 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3dec5a-c686-4248-9b8d-c2da3e885458 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:32.861315 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-8d3dec5a-c686-4248-9b8d-c2da3e885458 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:32.966635 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3dec5a-c686-4248-9b8d-c2da3e885458 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:01:32.968392 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3dec5a-c686-4248-9b8d-c2da3e885458 service nova] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:01:32.992632 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 698/1368] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:01:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:01:34.002878 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c83b5b-8008-4b4b-975a-43063dbb599b None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:34.003419 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c83b5b-8008-4b4b-975a-43063dbb599b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:34.003419 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c83b5b-8008-4b4b-975a-43063dbb599b None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:34.004995 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 671/1369] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 14:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:34.010073 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d592dd-eacd-4798-ba86-01ecfdbe2de6 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:34.010833 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d592dd-eacd-4798-ba86-01ecfdbe2de6 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:34.011279 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d592dd-eacd-4798-ba86-01ecfdbe2de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:34.047495 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:34.092210 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-89d592dd-eacd-4798-ba86-01ecfdbe2de6 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:34.093473 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-89d592dd-eacd-4798-ba86-01ecfdbe2de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:34.200201 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 699/1370] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 14:01:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:34.542776 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f76af-82ed-4781-ba3a-39fb283d136a None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:34.542776 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f76af-82ed-4781-ba3a-39fb283d136a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:34.542776 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f76af-82ed-4781-ba3a-39fb283d136a None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:34.547412 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 672/1371] 10.222.0.48 () {54 vars in 964 bytes} [Mon Aug 1 14:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:34.551767 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c54f1-4b5b-46fd-a851-cecffdca19ff None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:34.552060 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c54f1-4b5b-46fd-a851-cecffdca19ff None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:34.552346 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c54f1-4b5b-46fd-a851-cecffdca19ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:34.638573 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-846c54f1-4b5b-46fd-a851-cecffdca19ff None None] MFA Rules not processed for user `b185b052f9df4d3f9847d8f9b4cfda5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:34.639804 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-846c54f1-4b5b-46fd-a851-cecffdca19ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:34.690093 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:34.725080 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 700/1372] 10.222.0.48 () {58 vars in 1061 bytes} [Mon Aug 1 14:01:34 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:01:34.733318 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f76af-82ed-4781-ba3a-39fb283d136a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:01:34.734837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fb5f76af-82ed-4781-ba3a-39fb283d136a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:34.793290 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:34.861926 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f76af-82ed-4781-ba3a-39fb283d136a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:01:34.942756 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0dd559-4256-42df-816a-006554e4dafe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:01:34.943591 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0dd559-4256-42df-816a-006554e4dafe service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:34.943982 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0dd559-4256-42df-816a-006554e4dafe service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:34.944364 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0dd559-4256-42df-816a-006554e4dafe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:34.946203 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-de0dd559-4256-42df-816a-006554e4dafe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:35.063961 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:35.064674 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca4015-99f4-4973-9ca0-9c82c6d49a57 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:35.064674 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca4015-99f4-4973-9ca0-9c82c6d49a57 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:35.064674 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca4015-99f4-4973-9ca0-9c82c6d49a57 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:35.066091 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 701/1373] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 14:01:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:01:35.070925 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2d598-97fb-42fa-afdd-47812fa427be None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:35.071212 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2d598-97fb-42fa-afdd-47812fa427be None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:35.071503 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2d598-97fb-42fa-afdd-47812fa427be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:35.079075 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0dd559-4256-42df-816a-006554e4dafe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:01:35.080703 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0dd559-4256-42df-816a-006554e4dafe service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:01:35.099877 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 673/1374] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:01:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:01:35.142765 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-9cb2d598-97fb-42fa-afdd-47812fa427be None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:35.143969 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-9cb2d598-97fb-42fa-afdd-47812fa427be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:35.258957 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 702/1375] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 14:01:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:01:35.292265 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0dd559-4256-42df-816a-006554e4dafe service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:01:35.293502 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-de0dd559-4256-42df-816a-006554e4dafe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:35.418911 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0dd559-4256-42df-816a-006554e4dafe service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:01:35.552604 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:35.569727 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e04071-2177-46d6-b903-edff35d414b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:01:35.570584 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e04071-2177-46d6-b903-edff35d414b7 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:35.571008 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e04071-2177-46d6-b903-edff35d414b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:35.571473 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e04071-2177-46d6-b903-edff35d414b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:35.573749 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-80e04071-2177-46d6-b903-edff35d414b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:35.696962 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e04071-2177-46d6-b903-edff35d414b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:01:35.698568 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e04071-2177-46d6-b903-edff35d414b7 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:01:35.717459 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 674/1376] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:01:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:01:37.763663 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea6ce9-4a44-4c3c-a212-f04bfa977967 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:37.764190 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea6ce9-4a44-4c3c-a212-f04bfa977967 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:37.764190 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea6ce9-4a44-4c3c-a212-f04bfa977967 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:37.765639 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 703/1377] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 14:01:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:37.770781 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c270c4-8890-4a63-bb8e-929ee44dc951 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:37.771177 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c270c4-8890-4a63-bb8e-929ee44dc951 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:37.771288 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c270c4-8890-4a63-bb8e-929ee44dc951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:37.847446 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-90c270c4-8890-4a63-bb8e-929ee44dc951 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:37.848693 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-90c270c4-8890-4a63-bb8e-929ee44dc951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:37.909468 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:37.948592 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 675/1378] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 14:01:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:37.993705 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea6ce9-4a44-4c3c-a212-f04bfa977967 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:01:37.995252 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-8cea6ce9-4a44-4c3c-a212-f04bfa977967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:38.047830 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:38.177813 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea6ce9-4a44-4c3c-a212-f04bfa977967 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:01:38.291135 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e0b75-bae0-40aa-94b6-8cf07543d3a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:01:38.291675 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e0b75-bae0-40aa-94b6-8cf07543d3a1 service nova] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:38.291954 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e0b75-bae0-40aa-94b6-8cf07543d3a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:38.292229 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e0b75-bae0-40aa-94b6-8cf07543d3a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:38.293859 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-b93e0b75-bae0-40aa-94b6-8cf07543d3a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:38.305087 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:38.383362 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93e0b75-bae0-40aa-94b6-8cf07543d3a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:01:38.384998 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93e0b75-bae0-40aa-94b6-8cf07543d3a1 service nova] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:01:38.406344 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 704/1379] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:01:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:01:40.459718 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754f7b6-5fd6-431b-b14f-e142083a8cac None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:40.460269 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754f7b6-5fd6-431b-b14f-e142083a8cac None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:40.460269 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754f7b6-5fd6-431b-b14f-e142083a8cac None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:40.461785 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 676/1380] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 14:01:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:40.467866 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb82f4-af3c-4f9a-af27-ab264b8cfe39 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:40.471543 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb82f4-af3c-4f9a-af27-ab264b8cfe39 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:40.471543 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb82f4-af3c-4f9a-af27-ab264b8cfe39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:40.540290 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-49bb82f4-af3c-4f9a-af27-ab264b8cfe39 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:40.541782 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-49bb82f4-af3c-4f9a-af27-ab264b8cfe39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:40.638391 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:40.651837 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 705/1381] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 14:01:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:40.698689 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754f7b6-5fd6-431b-b14f-e142083a8cac None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:01:40.700264 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a754f7b6-5fd6-431b-b14f-e142083a8cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:40.867505 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754f7b6-5fd6-431b-b14f-e142083a8cac None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:01:40.919090 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:40.991367 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d14a26-85ae-47c9-9b75-18471b0562e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:01:40.991367 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d14a26-85ae-47c9-9b75-18471b0562e4 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:40.991367 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d14a26-85ae-47c9-9b75-18471b0562e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:40.992803 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d14a26-85ae-47c9-9b75-18471b0562e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:40.993574 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f4d14a26-85ae-47c9-9b75-18471b0562e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:41.089719 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d14a26-85ae-47c9-9b75-18471b0562e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:01:41.091367 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d14a26-85ae-47c9-9b75-18471b0562e4 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:01:41.109966 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 677/1382] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:01:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:01:43.144944 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9935c08-47cb-4f58-aefb-3b573a33406e None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:43.145525 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9935c08-47cb-4f58-aefb-3b573a33406e None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:43.145525 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9935c08-47cb-4f58-aefb-3b573a33406e None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:43.146845 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 706/1383] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 14:01:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:43.152032 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a9788-76ac-4f7b-b285-820e0c035238 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:43.152421 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a9788-76ac-4f7b-b285-820e0c035238 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:43.152607 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a9788-76ac-4f7b-b285-820e0c035238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:43.219069 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-611a9788-76ac-4f7b-b285-820e0c035238 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:43.220274 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-611a9788-76ac-4f7b-b285-820e0c035238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:43.222392 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:43.319567 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 678/1384] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 14:01:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:43.362195 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9935c08-47cb-4f58-aefb-3b573a33406e None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:01:43.363717 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-b9935c08-47cb-4f58-aefb-3b573a33406e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:43.517385 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9935c08-47cb-4f58-aefb-3b573a33406e None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:01:43.581202 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:43.632796 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef772f1-02bd-427e-8af5-76df17a8faee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:01:43.633419 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef772f1-02bd-427e-8af5-76df17a8faee service nova] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:43.633708 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef772f1-02bd-427e-8af5-76df17a8faee service nova] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:43.633994 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef772f1-02bd-427e-8af5-76df17a8faee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:43.635636 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-4ef772f1-02bd-427e-8af5-76df17a8faee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:43.743640 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef772f1-02bd-427e-8af5-76df17a8faee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:01:43.745280 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef772f1-02bd-427e-8af5-76df17a8faee service nova] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:01:43.763506 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 707/1385] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:01:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:01:46.368640 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae69e79-b659-44c8-9e27-d0052e4bb796 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:46.369202 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae69e79-b659-44c8-9e27-d0052e4bb796 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:46.369202 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae69e79-b659-44c8-9e27-d0052e4bb796 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:46.370703 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 679/1386] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 14:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:46.377283 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cf37d-8264-45e7-8afc-cf671bfd3170 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:46.377693 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cf37d-8264-45e7-8afc-cf671bfd3170 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:46.377856 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cf37d-8264-45e7-8afc-cf671bfd3170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:46.449131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-116cf37d-8264-45e7-8afc-cf671bfd3170 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:46.450348 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-116cf37d-8264-45e7-8afc-cf671bfd3170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:46.477864 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:46.574106 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 708/1387] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 14:01:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:46.622618 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae69e79-b659-44c8-9e27-d0052e4bb796 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:01:46.624091 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fae69e79-b659-44c8-9e27-d0052e4bb796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:46.762985 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:46.780747 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae69e79-b659-44c8-9e27-d0052e4bb796 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:01:46.903609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2149c9e9-e976-46e1-9a8b-010d87c2238a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:01:46.904249 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149c9e9-e976-46e1-9a8b-010d87c2238a service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:46.904560 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149c9e9-e976-46e1-9a8b-010d87c2238a service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:46.904837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149c9e9-e976-46e1-9a8b-010d87c2238a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:46.906496 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2149c9e9-e976-46e1-9a8b-010d87c2238a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:47.001081 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2149c9e9-e976-46e1-9a8b-010d87c2238a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:01:47.002680 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2149c9e9-e976-46e1-9a8b-010d87c2238a service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:01:47.007758 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:47.022016 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 680/1388] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:01:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:01:49.113565 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad074b63-2901-469f-8a49-ecf30794b90c None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:49.114070 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad074b63-2901-469f-8a49-ecf30794b90c None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:49.114070 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad074b63-2901-469f-8a49-ecf30794b90c None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:49.115708 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 709/1389] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 14:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:49.121073 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44fa88-4a1f-4004-bd47-08dc940dad3b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:49.121536 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44fa88-4a1f-4004-bd47-08dc940dad3b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:49.121822 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44fa88-4a1f-4004-bd47-08dc940dad3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:49.188212 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-fe44fa88-4a1f-4004-bd47-08dc940dad3b None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:49.189447 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fe44fa88-4a1f-4004-bd47-08dc940dad3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:49.286373 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 681/1390] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 14:01:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:49.329778 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad074b63-2901-469f-8a49-ecf30794b90c None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:01:49.331263 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-ad074b63-2901-469f-8a49-ecf30794b90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:49.488626 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad074b63-2901-469f-8a49-ecf30794b90c None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:01:49.497909 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:49.602802 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cf3fbe-ae21-4920-8405-8f012cf31a89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:01:49.603427 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf3fbe-ae21-4920-8405-8f012cf31a89 service nova] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:49.603709 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf3fbe-ae21-4920-8405-8f012cf31a89 service nova] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:49.603984 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf3fbe-ae21-4920-8405-8f012cf31a89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:49.605622 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-80cf3fbe-ae21-4920-8405-8f012cf31a89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:49.695906 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cf3fbe-ae21-4920-8405-8f012cf31a89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:01:49.697536 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cf3fbe-ae21-4920-8405-8f012cf31a89 service nova] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:01:49.717805 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 710/1391] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:01:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:01:50.921738 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c54d0-78b1-46af-978a-c5f831a98622 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:50.922325 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c54d0-78b1-46af-978a-c5f831a98622 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:50.922325 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c54d0-78b1-46af-978a-c5f831a98622 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:50.923659 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 682/1392] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 14:01:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:50.928851 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d00ee6-10fb-4b15-a340-1b38a370ea96 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:50.929298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d00ee6-10fb-4b15-a340-1b38a370ea96 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:50.929429 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d00ee6-10fb-4b15-a340-1b38a370ea96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:50.944823 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:50.999113 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-b5d00ee6-10fb-4b15-a340-1b38a370ea96 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:51.000297 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-b5d00ee6-10fb-4b15-a340-1b38a370ea96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:51.110311 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 711/1393] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 14:01:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:51.179272 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad4a8a-88f5-4cd4-925d-b619419ed481 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:51.179558 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad4a8a-88f5-4cd4-925d-b619419ed481 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:51.179844 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad4a8a-88f5-4cd4-925d-b619419ed481 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:51.181243 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 683/1394] 10.222.0.48 () {54 vars in 966 bytes} [Mon Aug 1 14:01:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:51.186382 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357aea6-8234-479e-877a-384b2925c081 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:51.186740 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357aea6-8234-479e-877a-384b2925c081 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:51.187087 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357aea6-8234-479e-877a-384b2925c081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:51.261600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-b357aea6-8234-479e-877a-384b2925c081 None None] MFA Rules not processed for user `39a66f7362964389be039937a786bb0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:51.263309 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-b357aea6-8234-479e-877a-384b2925c081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:51.298673 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:51.341677 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 712/1395] 10.222.0.48 () {58 vars in 1063 bytes} [Mon Aug 1 14:01:51 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:51.349227 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad4a8a-88f5-4cd4-925d-b619419ed481 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:01:51.350729 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-14ad4a8a-88f5-4cd4-925d-b619419ed481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:51.423506 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:51.456328 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad4a8a-88f5-4cd4-925d-b619419ed481 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:01:51.531396 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e7c5cc-306d-401e-8b2a-e0c98876151a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39a66f7362964389be039937a786bb0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:01:51.532015 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7c5cc-306d-401e-8b2a-e0c98876151a service glance] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:51.532296 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7c5cc-306d-401e-8b2a-e0c98876151a service glance] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:51.532601 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7c5cc-306d-401e-8b2a-e0c98876151a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:51.534197 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f1e7c5cc-306d-401e-8b2a-e0c98876151a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:51.643134 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e7c5cc-306d-401e-8b2a-e0c98876151a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:01:51.644592 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e7c5cc-306d-401e-8b2a-e0c98876151a service glance] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:01:51.659656 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:51.664807 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 684/1396] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:01:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:01:51.768852 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545384a4-f6b8-4672-8fa9-4d223b0805d4 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:51.769070 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545384a4-f6b8-4672-8fa9-4d223b0805d4 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:51.769358 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545384a4-f6b8-4672-8fa9-4d223b0805d4 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:51.770721 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 713/1397] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 14:01:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:51.776106 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71366a33-35dd-406e-b056-ed7a32943338 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:51.776396 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71366a33-35dd-406e-b056-ed7a32943338 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:51.776705 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71366a33-35dd-406e-b056-ed7a32943338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:51.845570 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-71366a33-35dd-406e-b056-ed7a32943338 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:51.846809 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-71366a33-35dd-406e-b056-ed7a32943338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:51.957273 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 685/1398] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 14:01:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:52.003192 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545384a4-f6b8-4672-8fa9-4d223b0805d4 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:01:52.004748 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-545384a4-f6b8-4672-8fa9-4d223b0805d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:52.166228 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545384a4-f6b8-4672-8fa9-4d223b0805d4 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:01:52.207226 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:52.288670 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d2b7e3-bc3d-4a69-8826-b17422fc92c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:01:52.289217 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2b7e3-bc3d-4a69-8826-b17422fc92c6 service nova] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:52.289502 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2b7e3-bc3d-4a69-8826-b17422fc92c6 service nova] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:52.289781 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2b7e3-bc3d-4a69-8826-b17422fc92c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:52.291763 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-e5d2b7e3-bc3d-4a69-8826-b17422fc92c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:52.388532 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d2b7e3-bc3d-4a69-8826-b17422fc92c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:01:52.390285 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d2b7e3-bc3d-4a69-8826-b17422fc92c6 service nova] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:01:52.397501 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407113a-55ca-44c6-8dfc-793158b4511b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:52.397798 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407113a-55ca-44c6-8dfc-793158b4511b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:52.398076 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407113a-55ca-44c6-8dfc-793158b4511b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:52.408815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 714/1399] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:01:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:01:52.466938 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c407113a-55ca-44c6-8dfc-793158b4511b None None] MFA Rules not processed for user `5007a2f8e6244b0788f0a554cc572fe5`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:52.468183 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c407113a-55ca-44c6-8dfc-793158b4511b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:52.503889 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:52.573149 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 686/1400] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 14:01:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:01:52.619650 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33585db3-6178-42a4-97e6-bb0781916676 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:52.619957 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33585db3-6178-42a4-97e6-bb0781916676 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:52.620226 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33585db3-6178-42a4-97e6-bb0781916676 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:52.621700 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 715/1401] 10.222.0.48 () {54 vars in 943 bytes} [Mon Aug 1 14:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:52.628848 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ec5bb-ba6d-4de1-b2f3-7ba65cebaa36 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:52.629172 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ec5bb-ba6d-4de1-b2f3-7ba65cebaa36 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:52.629453 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ec5bb-ba6d-4de1-b2f3-7ba65cebaa36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:52.708148 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-cb5ec5bb-ba6d-4de1-b2f3-7ba65cebaa36 None None] MFA Rules not processed for user `2fe2a6d7bd10435ba2c8e552d808b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:52.709463 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cb5ec5bb-ba6d-4de1-b2f3-7ba65cebaa36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:52.826135 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 687/1402] 10.222.0.48 () {58 vars in 1040 bytes} [Mon Aug 1 14:01:52 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:52.835447 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33585db3-6178-42a4-97e6-bb0781916676 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:01:52.837098 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-33585db3-6178-42a4-97e6-bb0781916676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:52.892036 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:52.979701 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33585db3-6178-42a4-97e6-bb0781916676 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:01:53.083637 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe09e2d-b46b-498f-a618-e09c0402e5dd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2a6d7bd10435ba2c8e552d808b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:01:53.084304 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe09e2d-b46b-498f-a618-e09c0402e5dd service swift] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:53.084448 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe09e2d-b46b-498f-a618-e09c0402e5dd service swift] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:53.084757 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe09e2d-b46b-498f-a618-e09c0402e5dd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:53.086461 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-8fe09e2d-b46b-498f-a618-e09c0402e5dd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:53.116829 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:53.171091 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe09e2d-b46b-498f-a618-e09c0402e5dd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:01:53.172815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe09e2d-b46b-498f-a618-e09c0402e5dd service swift] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:01:53.180799 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 716/1403] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 14:01:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:01:54.530043 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d402a-1826-4f87-b08d-24f9ae3fb068 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:54.530612 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d402a-1826-4f87-b08d-24f9ae3fb068 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:54.530612 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d402a-1826-4f87-b08d-24f9ae3fb068 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:54.536096 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 688/1404] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 14:01:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:54.537461 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a06369-f494-4475-bb6b-c6dbd3cb2dcb None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:54.537936 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a06369-f494-4475-bb6b-c6dbd3cb2dcb None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:54.538158 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a06369-f494-4475-bb6b-c6dbd3cb2dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:54.608378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-03a06369-f494-4475-bb6b-c6dbd3cb2dcb None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:54.609965 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-03a06369-f494-4475-bb6b-c6dbd3cb2dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:54.717613 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 717/1405] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 14:01:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:01:54.763469 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d402a-1826-4f87-b08d-24f9ae3fb068 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:01:54.765037 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ee0d402a-1826-4f87-b08d-24f9ae3fb068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:54.767987 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:54.959622 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d402a-1826-4f87-b08d-24f9ae3fb068 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:01:55.051353 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:55.097254 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b01e5-0c18-493f-ada5-0c271b7377be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:01:55.097798 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b01e5-0c18-493f-ada5-0c271b7377be service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:55.098084 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b01e5-0c18-493f-ada5-0c271b7377be service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:55.098370 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b01e5-0c18-493f-ada5-0c271b7377be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:55.099994 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-087b01e5-0c18-493f-ada5-0c271b7377be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:55.203352 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b01e5-0c18-493f-ada5-0c271b7377be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:01:55.204398 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b01e5-0c18-493f-ada5-0c271b7377be service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:01:55.226050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 689/1406] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:01:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:01:57.342047 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514471d4-8e96-49e2-9adf-f29f775c819c None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:57.342612 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514471d4-8e96-49e2-9adf-f29f775c819c None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:57.342612 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514471d4-8e96-49e2-9adf-f29f775c819c None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:57.344048 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 718/1407] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 14:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:01:57.349374 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb4b4c-2573-4cd1-b5b3-b796d8228e77 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:57.349797 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb4b4c-2573-4cd1-b5b3-b796d8228e77 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:57.349938 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb4b4c-2573-4cd1-b5b3-b796d8228e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:57.421244 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f3bb4b4c-2573-4cd1-b5b3-b796d8228e77 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:01:57.421788 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f3bb4b4c-2573-4cd1-b5b3-b796d8228e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:57.464100 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:57.532044 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 690/1408] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 14:01:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:01:57.577977 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514471d4-8e96-49e2-9adf-f29f775c819c None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:01:57.579500 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-514471d4-8e96-49e2-9adf-f29f775c819c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:57.603144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:57.750395 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514471d4-8e96-49e2-9adf-f29f775c819c None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:01:57.866941 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:01:57.893482 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052de83d-5ebb-407d-8af5-c43970043d18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:01:57.894170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052de83d-5ebb-407d-8af5-c43970043d18 service nova] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:01:57.894445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052de83d-5ebb-407d-8af5-c43970043d18 service nova] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:01:57.894712 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052de83d-5ebb-407d-8af5-c43970043d18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:01:57.896953 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-052de83d-5ebb-407d-8af5-c43970043d18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:01:58.014398 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052de83d-5ebb-407d-8af5-c43970043d18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:01:58.016589 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052de83d-5ebb-407d-8af5-c43970043d18 service nova] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:01:58.039742 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 719/1409] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:01:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:02:07.530287 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=26 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:08.036579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=52 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:15.811319 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b56c4-5222-4ef7-8ab5-2eb53666809e None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:15.811898 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b56c4-5222-4ef7-8ab5-2eb53666809e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:15.811898 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b56c4-5222-4ef7-8ab5-2eb53666809e None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:15.813282 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 691/1410] 10.222.0.48 () {54 vars in 918 bytes} [Mon Aug 1 14:02:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:02:15.818549 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c37e8-ebf4-4ae3-a8f3-d5d0138c1ebb None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:15.818940 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c37e8-ebf4-4ae3-a8f3-d5d0138c1ebb None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:15.819135 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c37e8-ebf4-4ae3-a8f3-d5d0138c1ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:15.893563 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-434c37e8-ebf4-4ae3-a8f3-d5d0138c1ebb None None] MFA Rules not processed for user `e294fe796a1a4ac89041a532856ad627`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:02:15.894769 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-434c37e8-ebf4-4ae3-a8f3-d5d0138c1ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:16.001226 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 720/1411] 10.222.0.48 () {58 vars in 1015 bytes} [Mon Aug 1 14:02:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:02:16.016077 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5968a-794a-43f3-9590-4240002c1a29 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:16.016473 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5968a-794a-43f3-9590-4240002c1a29 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:16.017022 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5968a-794a-43f3-9590-4240002c1a29 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:16.018970 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 692/1412] 10.222.0.48 () {54 vars in 918 bytes} [Mon Aug 1 14:02:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:02:16.023919 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab29d59-fca7-431e-a913-9e2cfaa551b3 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:16.024213 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab29d59-fca7-431e-a913-9e2cfaa551b3 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:16.024669 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab29d59-fca7-431e-a913-9e2cfaa551b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:16.096566 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:16.099606 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-5ab29d59-fca7-431e-a913-9e2cfaa551b3 None None] MFA Rules not processed for user `a246654c783b4cc68d212e91bb05d8c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:02:16.101348 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-5ab29d59-fca7-431e-a913-9e2cfaa551b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:16.216778 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 721/1413] 10.222.0.48 () {58 vars in 1015 bytes} [Mon Aug 1 14:02:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:02:18.655722 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2671d8-7338-4d6b-82d3-e0c9d7ddfdfe None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:18.656259 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2671d8-7338-4d6b-82d3-e0c9d7ddfdfe None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:18.656382 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2671d8-7338-4d6b-82d3-e0c9d7ddfdfe None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:18.657732 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 693/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:02:18.663046 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9d426-73ca-4b38-9cd5-f6bcfa8a045a None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:18.663442 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9d426-73ca-4b38-9cd5-f6bcfa8a045a None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:18.663602 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9d426-73ca-4b38-9cd5-f6bcfa8a045a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:18.729704 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-36a9d426-73ca-4b38-9cd5-f6bcfa8a045a None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:02:18.730911 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-36a9d426-73ca-4b38-9cd5-f6bcfa8a045a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:18.786256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:18.828675 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 722/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:02:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:02:18.871119 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2671d8-7338-4d6b-82d3-e0c9d7ddfdfe None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:02:18.872676 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1f2671d8-7338-4d6b-82d3-e0c9d7ddfdfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:19.022633 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2671d8-7338-4d6b-82d3-e0c9d7ddfdfe None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:02:19.096147 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:19.136628 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec19da60-14cf-4a7c-a08e-2972559e6e5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:02:19.137249 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19da60-14cf-4a7c-a08e-2972559e6e5f service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:19.137533 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19da60-14cf-4a7c-a08e-2972559e6e5f service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:19.137810 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19da60-14cf-4a7c-a08e-2972559e6e5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:19.139439 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ec19da60-14cf-4a7c-a08e-2972559e6e5f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:19.227458 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec19da60-14cf-4a7c-a08e-2972559e6e5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:02:19.229705 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec19da60-14cf-4a7c-a08e-2972559e6e5f service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:02:19.247588 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 694/1416] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:02:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:02:20.141041 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc4f48-23f5-4f21-8826-c656bea2fb4b None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:20.141553 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc4f48-23f5-4f21-8826-c656bea2fb4b None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:20.141553 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc4f48-23f5-4f21-8826-c656bea2fb4b None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:20.142871 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 723/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 14:02:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:02:20.147799 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebf9c2-b6e0-4f5b-9bbb-d760d0a4a373 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:20.148213 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebf9c2-b6e0-4f5b-9bbb-d760d0a4a373 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:20.148355 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebf9c2-b6e0-4f5b-9bbb-d760d0a4a373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:20.227007 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c3ebf9c2-b6e0-4f5b-9bbb-d760d0a4a373 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:02:20.228247 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c3ebf9c2-b6e0-4f5b-9bbb-d760d0a4a373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:20.255444 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:20.325115 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 695/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 14:02:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:02:20.483450 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc4f48-23f5-4f21-8826-c656bea2fb4b None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:02:20.485017 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-fffc4f48-23f5-4f21-8826-c656bea2fb4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:20.630466 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc4f48-23f5-4f21-8826-c656bea2fb4b None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:02:20.672970 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:20.744975 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68000f77-99e9-4fac-93c8-e2560c28aff0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:02:20.745507 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68000f77-99e9-4fac-93c8-e2560c28aff0 service nova] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:20.745782 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68000f77-99e9-4fac-93c8-e2560c28aff0 service nova] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:20.746052 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68000f77-99e9-4fac-93c8-e2560c28aff0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:20.747641 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-68000f77-99e9-4fac-93c8-e2560c28aff0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:20.834505 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68000f77-99e9-4fac-93c8-e2560c28aff0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:02:20.836097 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68000f77-99e9-4fac-93c8-e2560c28aff0 service nova] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:02:20.854706 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 724/1419] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:02:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:02:21.743018 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae8accb-f570-45bc-8fa2-43f1521ae67b None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:21.743520 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae8accb-f570-45bc-8fa2-43f1521ae67b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:21.743636 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae8accb-f570-45bc-8fa2-43f1521ae67b None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:21.745482 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 696/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 14:02:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:02:21.750547 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6715de4-7945-40bc-b5b6-33ff1a9342c5 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:21.750973 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6715de4-7945-40bc-b5b6-33ff1a9342c5 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:21.751104 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6715de4-7945-40bc-b5b6-33ff1a9342c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:21.804535 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:21.827042 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-c6715de4-7945-40bc-b5b6-33ff1a9342c5 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:02:21.828248 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-c6715de4-7945-40bc-b5b6-33ff1a9342c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:21.923182 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 725/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 14:02:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:02:22.075793 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae8accb-f570-45bc-8fa2-43f1521ae67b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:02:22.077348 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-dae8accb-f570-45bc-8fa2-43f1521ae67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:22.225518 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae8accb-f570-45bc-8fa2-43f1521ae67b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:02:22.239201 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:22.340504 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc52d8c5-48de-4ec3-bb16-4ad3d363f6a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:02:22.341147 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52d8c5-48de-4ec3-bb16-4ad3d363f6a9 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:22.341434 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52d8c5-48de-4ec3-bb16-4ad3d363f6a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:22.341718 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52d8c5-48de-4ec3-bb16-4ad3d363f6a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:22.343518 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fc52d8c5-48de-4ec3-bb16-4ad3d363f6a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:22.448868 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc52d8c5-48de-4ec3-bb16-4ad3d363f6a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:02:22.450477 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc52d8c5-48de-4ec3-bb16-4ad3d363f6a9 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:02:22.468838 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 697/1422] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:02:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:02:23.336068 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc27931-4f60-461b-b003-ee5acbc4b4b2 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:23.336662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc27931-4f60-461b-b003-ee5acbc4b4b2 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:23.336662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc27931-4f60-461b-b003-ee5acbc4b4b2 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:23.338043 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 726/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 14:02:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:02:23.343464 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae526d2-40d6-47d5-95d1-10bcec401b1a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:23.343979 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae526d2-40d6-47d5-95d1-10bcec401b1a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:23.344252 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae526d2-40d6-47d5-95d1-10bcec401b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:23.364518 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:23.429861 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-0ae526d2-40d6-47d5-95d1-10bcec401b1a None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:02:23.431098 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0ae526d2-40d6-47d5-95d1-10bcec401b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:23.527492 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 698/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 14:02:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:02:23.677521 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc27931-4f60-461b-b003-ee5acbc4b4b2 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:02:23.679032 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-cbc27931-4f60-461b-b003-ee5acbc4b4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:23.787532 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:23.825124 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc27931-4f60-461b-b003-ee5acbc4b4b2 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:02:23.941577 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d142a2df-5942-417d-af97-e5f9337fda12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:02:23.942117 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142a2df-5942-417d-af97-e5f9337fda12 service nova] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:23.942406 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142a2df-5942-417d-af97-e5f9337fda12 service nova] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:23.942682 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142a2df-5942-417d-af97-e5f9337fda12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:23.944319 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-d142a2df-5942-417d-af97-e5f9337fda12 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:24.019634 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:24.034352 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d142a2df-5942-417d-af97-e5f9337fda12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:02:24.035965 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d142a2df-5942-417d-af97-e5f9337fda12 service nova] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:02:24.053858 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 727/1425] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:02:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:02:24.917721 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d971d9-e2ff-4634-970f-36e8c4b72e2a None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:24.918276 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d971d9-e2ff-4634-970f-36e8c4b72e2a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:24.918276 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d971d9-e2ff-4634-970f-36e8c4b72e2a None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:24.919644 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 699/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 14:02:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:02:24.924405 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7225f232-6a26-49b6-956e-479184efc891 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:24.924871 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7225f232-6a26-49b6-956e-479184efc891 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:24.925039 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7225f232-6a26-49b6-956e-479184efc891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:25.001169 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-7225f232-6a26-49b6-956e-479184efc891 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:02:25.002374 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-7225f232-6a26-49b6-956e-479184efc891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:25.096952 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 728/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 14:02:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:02:25.252831 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d971d9-e2ff-4634-970f-36e8c4b72e2a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:02:25.254366 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e8d971d9-e2ff-4634-970f-36e8c4b72e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:25.339142 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:25.401800 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d971d9-e2ff-4634-970f-36e8c4b72e2a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:02:25.514320 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907cf32b-270e-4eb4-8184-3f6d5e508957 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:02:25.514938 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907cf32b-270e-4eb4-8184-3f6d5e508957 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:25.515222 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907cf32b-270e-4eb4-8184-3f6d5e508957 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:25.515496 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907cf32b-270e-4eb4-8184-3f6d5e508957 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:25.517146 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-907cf32b-270e-4eb4-8184-3f6d5e508957 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:25.564728 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:25.606313 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907cf32b-270e-4eb4-8184-3f6d5e508957 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:02:25.607914 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907cf32b-270e-4eb4-8184-3f6d5e508957 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:02:25.625862 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 700/1428] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:02:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:02:26.420245 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6372a3-8d03-4af2-8a56-7a6519f1cb27 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:26.420821 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6372a3-8d03-4af2-8a56-7a6519f1cb27 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:26.420936 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6372a3-8d03-4af2-8a56-7a6519f1cb27 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:26.422334 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 729/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 14:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:02:26.427254 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a75d321-ecbc-4374-bd69-33a494c799c0 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:26.427696 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a75d321-ecbc-4374-bd69-33a494c799c0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:26.427802 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a75d321-ecbc-4374-bd69-33a494c799c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:26.508731 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-6a75d321-ecbc-4374-bd69-33a494c799c0 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:02:26.509994 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6a75d321-ecbc-4374-bd69-33a494c799c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:26.607265 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 701/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 14:02:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:02:26.760302 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6372a3-8d03-4af2-8a56-7a6519f1cb27 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:02:26.761725 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-ca6372a3-8d03-4af2-8a56-7a6519f1cb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:26.809956 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:26.908563 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6372a3-8d03-4af2-8a56-7a6519f1cb27 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:02:27.019997 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412a2993-ad89-46b6-bba9-726744cdc8b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:02:27.020561 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a2993-ad89-46b6-bba9-726744cdc8b3 service nova] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:27.020847 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a2993-ad89-46b6-bba9-726744cdc8b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:27.021127 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a2993-ad89-46b6-bba9-726744cdc8b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:27.022758 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-412a2993-ad89-46b6-bba9-726744cdc8b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:27.029635 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:27.111358 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412a2993-ad89-46b6-bba9-726744cdc8b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:02:27.112989 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412a2993-ad89-46b6-bba9-726744cdc8b3 service nova] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:02:27.131343 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 730/1431] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:02:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:02:27.931978 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7032f54-fdfd-49ab-b9a0-fffd87b118f1 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:27.932513 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7032f54-fdfd-49ab-b9a0-fffd87b118f1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:27.932513 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7032f54-fdfd-49ab-b9a0-fffd87b118f1 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:27.933899 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 702/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 14:02:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:02:27.938907 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde3ac3-0126-4a02-9f88-d1d3a8aadd7b None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:27.939332 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde3ac3-0126-4a02-9f88-d1d3a8aadd7b None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:27.939452 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde3ac3-0126-4a02-9f88-d1d3a8aadd7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:28.013536 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-6dde3ac3-0126-4a02-9f88-d1d3a8aadd7b None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:02:28.014736 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-6dde3ac3-0126-4a02-9f88-d1d3a8aadd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:28.086256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:28.116435 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 731/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 14:02:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:02:28.265352 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7032f54-fdfd-49ab-b9a0-fffd87b118f1 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:02:28.266786 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b7032f54-fdfd-49ab-b9a0-fffd87b118f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:28.284512 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:28.415780 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7032f54-fdfd-49ab-b9a0-fffd87b118f1 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:02:28.503194 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:28.649833 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ddb414-3939-4e82-83db-727792f15a3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:02:28.650444 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddb414-3939-4e82-83db-727792f15a3c service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:28.650721 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddb414-3939-4e82-83db-727792f15a3c service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:28.650995 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddb414-3939-4e82-83db-727792f15a3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:28.652610 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-88ddb414-3939-4e82-83db-727792f15a3c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:28.746058 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ddb414-3939-4e82-83db-727792f15a3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:02:28.747636 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ddb414-3939-4e82-83db-727792f15a3c service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:02:28.765436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 703/1434] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:02:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:02:29.562113 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8507d8-cbc5-45e1-acd3-7ab8a37a229e None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:29.562698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8507d8-cbc5-45e1-acd3-7ab8a37a229e None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:29.562698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8507d8-cbc5-45e1-acd3-7ab8a37a229e None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:29.564030 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 732/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 14:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:02:29.569056 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7c959-356e-4f27-92d9-8d1badb5616f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:29.569475 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7c959-356e-4f27-92d9-8d1badb5616f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:29.569612 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7c959-356e-4f27-92d9-8d1badb5616f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:29.644347 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-55f7c959-356e-4f27-92d9-8d1badb5616f None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:02:29.645562 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-55f7c959-356e-4f27-92d9-8d1badb5616f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:29.686203 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:29.739681 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 704/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 14:02:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:02:29.890156 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8507d8-cbc5-45e1-acd3-7ab8a37a229e None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:02:29.891674 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-8f8507d8-cbc5-45e1-acd3-7ab8a37a229e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:30.034986 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8507d8-cbc5-45e1-acd3-7ab8a37a229e None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:02:30.088817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:30.147171 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a4ba39-d470-4147-ba24-4254de0c7e93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:02:30.147802 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4ba39-d470-4147-ba24-4254de0c7e93 service nova] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:30.148088 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4ba39-d470-4147-ba24-4254de0c7e93 service nova] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:30.148362 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4ba39-d470-4147-ba24-4254de0c7e93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:30.150049 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-c8a4ba39-d470-4147-ba24-4254de0c7e93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:30.236974 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a4ba39-d470-4147-ba24-4254de0c7e93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:02:30.238803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a4ba39-d470-4147-ba24-4254de0c7e93 service nova] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:02:30.257011 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 733/1437] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:02:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:02:31.063697 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfbcf5a-77fa-4352-81f3-b3c045acb6fd None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:31.064229 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfbcf5a-77fa-4352-81f3-b3c045acb6fd None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:31.064322 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfbcf5a-77fa-4352-81f3-b3c045acb6fd None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:31.065644 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 705/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 14:02:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:02:31.070591 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162cb644-ee79-4160-ad06-b061d6caf157 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:31.071212 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162cb644-ee79-4160-ad06-b061d6caf157 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:31.071212 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162cb644-ee79-4160-ad06-b061d6caf157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:31.154742 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-162cb644-ee79-4160-ad06-b061d6caf157 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:02:31.155946 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-162cb644-ee79-4160-ad06-b061d6caf157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:31.157985 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:31.256260 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 734/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 14:02:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:02:31.407298 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfbcf5a-77fa-4352-81f3-b3c045acb6fd None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:02:31.408853 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-bcfbcf5a-77fa-4352-81f3-b3c045acb6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:31.563542 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfbcf5a-77fa-4352-81f3-b3c045acb6fd None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:02:31.590512 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:31.676064 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a33c974-a0eb-4ec6-a32b-157f6797a05a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:02:31.676710 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33c974-a0eb-4ec6-a32b-157f6797a05a service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:31.676996 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33c974-a0eb-4ec6-a32b-157f6797a05a service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:31.677278 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33c974-a0eb-4ec6-a32b-157f6797a05a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:31.678890 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5a33c974-a0eb-4ec6-a32b-157f6797a05a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:31.767297 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a33c974-a0eb-4ec6-a32b-157f6797a05a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:02:31.768915 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a33c974-a0eb-4ec6-a32b-157f6797a05a service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:02:31.786875 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 706/1440] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:02:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:02:33.691295 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c2a02-fa21-48d0-abd4-7dc9c9d01157 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:33.691699 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c2a02-fa21-48d0-abd4-7dc9c9d01157 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:33.691873 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c2a02-fa21-48d0-abd4-7dc9c9d01157 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:33.693279 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 735/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:02:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:02:33.698917 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a572a1-06f8-41dd-8258-69a0069fcfee None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:33.699333 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a572a1-06f8-41dd-8258-69a0069fcfee None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:33.699450 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a572a1-06f8-41dd-8258-69a0069fcfee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:33.740417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:33.765816 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-47a572a1-06f8-41dd-8258-69a0069fcfee None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:02:33.767042 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-47a572a1-06f8-41dd-8258-69a0069fcfee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:33.868726 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 707/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:02:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:02:33.906244 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c2a02-fa21-48d0-abd4-7dc9c9d01157 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:02:33.907812 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-ab5c2a02-fa21-48d0-abd4-7dc9c9d01157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:34.013967 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c2a02-fa21-48d0-abd4-7dc9c9d01157 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:02:34.046118 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:34.093839 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77c2ec-51e7-4781-8c42-bd0beae5d5e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:02:34.094464 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77c2ec-51e7-4781-8c42-bd0beae5d5e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:34.094750 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77c2ec-51e7-4781-8c42-bd0beae5d5e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:34.095029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77c2ec-51e7-4781-8c42-bd0beae5d5e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:34.096779 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-5f77c2ec-51e7-4781-8c42-bd0beae5d5e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:34.203739 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f77c2ec-51e7-4781-8c42-bd0beae5d5e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:02:34.205126 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f77c2ec-51e7-4781-8c42-bd0beae5d5e5 service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:02:34.223316 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 736/1443] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:02:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:02:36.373166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962bdfd9-9901-496d-8116-c5e445166e4d None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:36.373726 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962bdfd9-9901-496d-8116-c5e445166e4d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:36.373726 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962bdfd9-9901-496d-8116-c5e445166e4d None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:36.375071 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 708/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:02:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:02:36.380220 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc0362-26c1-47f7-8bc7-604e38891e94 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:36.380652 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc0362-26c1-47f7-8bc7-604e38891e94 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:36.380808 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc0362-26c1-47f7-8bc7-604e38891e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:36.451542 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-1dbc0362-26c1-47f7-8bc7-604e38891e94 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:02:36.452800 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-1dbc0362-26c1-47f7-8bc7-604e38891e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:36.470620 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:36.564182 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 737/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:02:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:02:36.603203 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962bdfd9-9901-496d-8116-c5e445166e4d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:02:36.604732 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-962bdfd9-9901-496d-8116-c5e445166e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:36.708472 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962bdfd9-9901-496d-8116-c5e445166e4d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:02:36.721150 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:36.782884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333706f9-91c6-48ca-9edd-42578e448cee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:02:36.783503 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333706f9-91c6-48ca-9edd-42578e448cee service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:36.783785 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333706f9-91c6-48ca-9edd-42578e448cee service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:36.784063 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333706f9-91c6-48ca-9edd-42578e448cee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:36.785714 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-333706f9-91c6-48ca-9edd-42578e448cee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:36.913461 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333706f9-91c6-48ca-9edd-42578e448cee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:02:36.914943 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333706f9-91c6-48ca-9edd-42578e448cee service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:02:36.933563 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 709/1446] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:02:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:02:39.282392 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae27fe-906a-4f89-a263-a4ea0e869f7f None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:39.282958 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae27fe-906a-4f89-a263-a4ea0e869f7f None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:39.282958 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae27fe-906a-4f89-a263-a4ea0e869f7f None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:39.284290 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 738/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:02:39.290132 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f3c04-2130-485f-a85a-e4c622523fb7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:39.290522 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f3c04-2130-485f-a85a-e4c622523fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:39.290681 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f3c04-2130-485f-a85a-e4c622523fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:39.337341 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:39.356159 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-2a0f3c04-2130-485f-a85a-e4c622523fb7 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:02:39.357455 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2a0f3c04-2130-485f-a85a-e4c622523fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:39.451321 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 710/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:02:39.489983 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae27fe-906a-4f89-a263-a4ea0e869f7f None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:02:39.491498 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-47ae27fe-906a-4f89-a263-a4ea0e869f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:39.594165 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae27fe-906a-4f89-a263-a4ea0e869f7f None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:02:39.633684 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:39.668426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592a48d-d799-4740-96b5-c7fcd8e45758 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:02:39.669080 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592a48d-d799-4740-96b5-c7fcd8e45758 service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:39.669367 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592a48d-d799-4740-96b5-c7fcd8e45758 service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:39.669648 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592a48d-d799-4740-96b5-c7fcd8e45758 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:39.671282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-3592a48d-d799-4740-96b5-c7fcd8e45758 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:39.900963 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3592a48d-d799-4740-96b5-c7fcd8e45758 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:02:39.902391 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3592a48d-d799-4740-96b5-c7fcd8e45758 service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:02:39.919682 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 739/1449] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:02:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:02:42.034643 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa98fd5-d5a8-42d1-8cb6-3b4a922078d1 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:42.035181 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa98fd5-d5a8-42d1-8cb6-3b4a922078d1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:42.035181 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa98fd5-d5a8-42d1-8cb6-3b4a922078d1 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:42.036456 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 711/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:02:42.042152 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3895c-a8f6-41ce-bfcf-809b24238eac None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:42.042540 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3895c-a8f6-41ce-bfcf-809b24238eac None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:42.042711 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3895c-a8f6-41ce-bfcf-809b24238eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:42.107134 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-bee3895c-a8f6-41ce-bfcf-809b24238eac None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:02:42.108339 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-bee3895c-a8f6-41ce-bfcf-809b24238eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:42.129395 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:42.204886 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 740/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:02:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:02:42.246342 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa98fd5-d5a8-42d1-8cb6-3b4a922078d1 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:02:42.248760 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-eaa98fd5-d5a8-42d1-8cb6-3b4a922078d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:42.352656 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa98fd5-d5a8-42d1-8cb6-3b4a922078d1 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:02:42.372398 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:42.425214 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ff9fe0-a367-43ff-9c40-6deeb7526a85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:02:42.425832 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff9fe0-a367-43ff-9c40-6deeb7526a85 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:42.426114 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff9fe0-a367-43ff-9c40-6deeb7526a85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:42.426393 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff9fe0-a367-43ff-9c40-6deeb7526a85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:42.428000 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-15ff9fe0-a367-43ff-9c40-6deeb7526a85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:42.542343 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ff9fe0-a367-43ff-9c40-6deeb7526a85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:02:42.543768 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ff9fe0-a367-43ff-9c40-6deeb7526a85 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:02:42.561120 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 712/1452] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:02:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:02:44.673712 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986afdd-af84-4076-8510-3047eb8152d9 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:44.674285 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986afdd-af84-4076-8510-3047eb8152d9 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:44.674285 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986afdd-af84-4076-8510-3047eb8152d9 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:44.675636 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 741/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:02:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:02:44.681252 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263ee82-158c-43e5-934e-a7b9e7b53176 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:44.681698 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263ee82-158c-43e5-934e-a7b9e7b53176 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:44.681831 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263ee82-158c-43e5-934e-a7b9e7b53176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:44.745438 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:44.748088 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e263ee82-158c-43e5-934e-a7b9e7b53176 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:02:44.749901 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e263ee82-158c-43e5-934e-a7b9e7b53176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:44.845495 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 713/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:02:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:02:44.883966 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986afdd-af84-4076-8510-3047eb8152d9 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:02:44.885517 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-d986afdd-af84-4076-8510-3047eb8152d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:44.991646 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986afdd-af84-4076-8510-3047eb8152d9 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:02:45.039963 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:45.064297 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab57a87-a7c2-4257-a41f-fc4205980f43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:02:45.064934 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab57a87-a7c2-4257-a41f-fc4205980f43 service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:45.065213 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab57a87-a7c2-4257-a41f-fc4205980f43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:45.065483 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab57a87-a7c2-4257-a41f-fc4205980f43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:45.067071 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-9ab57a87-a7c2-4257-a41f-fc4205980f43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:45.173129 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab57a87-a7c2-4257-a41f-fc4205980f43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:02:45.174541 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab57a87-a7c2-4257-a41f-fc4205980f43 service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:02:45.193740 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 742/1455] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:02:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:02:47.244585 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812ce48-5a40-42b5-9538-0bf1064106a4 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:47.245198 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812ce48-5a40-42b5-9538-0bf1064106a4 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:47.245198 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812ce48-5a40-42b5-9538-0bf1064106a4 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:47.246533 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 714/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:02:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:02:47.251794 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621916e-c9e6-4330-a440-d89694157a60 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:47.252215 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621916e-c9e6-4330-a440-d89694157a60 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:47.252361 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621916e-c9e6-4330-a440-d89694157a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:47.317244 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-1621916e-c9e6-4330-a440-d89694157a60 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:02:47.318446 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-1621916e-c9e6-4330-a440-d89694157a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:47.347871 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:47.414919 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 743/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:02:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:02:47.452901 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812ce48-5a40-42b5-9538-0bf1064106a4 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:02:47.454413 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f812ce48-5a40-42b5-9538-0bf1064106a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:47.556621 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812ce48-5a40-42b5-9538-0bf1064106a4 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:02:47.584971 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:47.629903 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af5a9e3-1321-4e74-8e3f-f7fd151a5820 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:02:47.630518 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5a9e3-1321-4e74-8e3f-f7fd151a5820 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:47.630802 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5a9e3-1321-4e74-8e3f-f7fd151a5820 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:47.631077 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5a9e3-1321-4e74-8e3f-f7fd151a5820 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:47.633333 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0af5a9e3-1321-4e74-8e3f-f7fd151a5820 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:47.740978 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af5a9e3-1321-4e74-8e3f-f7fd151a5820 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:02:47.742914 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af5a9e3-1321-4e74-8e3f-f7fd151a5820 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:02:47.761192 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 715/1458] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:02:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:02:49.808020 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cfdc7c-5187-4476-9c4d-58b8977309bc None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:49.808553 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cfdc7c-5187-4476-9c4d-58b8977309bc None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:49.808688 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cfdc7c-5187-4476-9c4d-58b8977309bc None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:49.809958 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 744/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:02:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:02:49.815389 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051f7dd-3a91-42fc-bd57-4e13d2a9f47a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:49.815856 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051f7dd-3a91-42fc-bd57-4e13d2a9f47a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:49.815961 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051f7dd-3a91-42fc-bd57-4e13d2a9f47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:49.882466 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d051f7dd-3a91-42fc-bd57-4e13d2a9f47a None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:02:49.883700 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d051f7dd-3a91-42fc-bd57-4e13d2a9f47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:49.892442 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:49.986465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 716/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:02:50.023672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cfdc7c-5187-4476-9c4d-58b8977309bc None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:02:50.025269 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-e9cfdc7c-5187-4476-9c4d-58b8977309bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:50.127777 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cfdc7c-5187-4476-9c4d-58b8977309bc None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:02:50.183788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:50.202579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7023ba5-9181-4a67-82c4-5ee5d947d97c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:02:50.203194 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7023ba5-9181-4a67-82c4-5ee5d947d97c service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:50.203477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7023ba5-9181-4a67-82c4-5ee5d947d97c service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:50.203755 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7023ba5-9181-4a67-82c4-5ee5d947d97c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:50.205397 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-c7023ba5-9181-4a67-82c4-5ee5d947d97c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:50.313945 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7023ba5-9181-4a67-82c4-5ee5d947d97c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:02:50.315416 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7023ba5-9181-4a67-82c4-5ee5d947d97c service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:02:50.333649 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 745/1461] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:02:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:02:52.472925 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1a38e-0a06-4ec1-b4ac-80dfce008c3f None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:52.473514 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1a38e-0a06-4ec1-b4ac-80dfce008c3f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:52.473514 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1a38e-0a06-4ec1-b4ac-80dfce008c3f None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:52.474838 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 717/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:02:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:02:52.480589 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a869592-b48b-49b8-bad9-828eb1025148 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:52.480992 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a869592-b48b-49b8-bad9-828eb1025148 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:52.481151 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a869592-b48b-49b8-bad9-828eb1025148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:52.548290 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-9a869592-b48b-49b8-bad9-828eb1025148 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:02:52.549530 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-9a869592-b48b-49b8-bad9-828eb1025148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:52.587339 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:52.646448 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 746/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:02:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:02:52.684154 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d1a38e-0a06-4ec1-b4ac-80dfce008c3f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:02:52.685723 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c6d1a38e-0a06-4ec1-b4ac-80dfce008c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:52.787659 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d1a38e-0a06-4ec1-b4ac-80dfce008c3f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:02:52.825903 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:52.862157 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4884ba-6c26-4cec-906e-e554ee96867d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:02:52.863021 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4884ba-6c26-4cec-906e-e554ee96867d service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:52.863297 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4884ba-6c26-4cec-906e-e554ee96867d service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:52.863569 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4884ba-6c26-4cec-906e-e554ee96867d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:52.865209 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cc4884ba-6c26-4cec-906e-e554ee96867d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:52.971483 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4884ba-6c26-4cec-906e-e554ee96867d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:02:52.972963 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4884ba-6c26-4cec-906e-e554ee96867d service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:02:52.991536 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 718/1464] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:02:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:02:55.065363 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970eebf-a9a4-4e06-8b75-fc5db0c06f5b None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:55.065875 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970eebf-a9a4-4e06-8b75-fc5db0c06f5b None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:55.065875 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970eebf-a9a4-4e06-8b75-fc5db0c06f5b None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:55.067256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 747/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:02:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:02:55.073188 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b5bd9-2ee3-4d91-beb9-c6799a5db3ec None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:55.073653 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b5bd9-2ee3-4d91-beb9-c6799a5db3ec None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:55.073752 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b5bd9-2ee3-4d91-beb9-c6799a5db3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:55.140879 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-1a8b5bd9-2ee3-4d91-beb9-c6799a5db3ec None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:02:55.142095 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1a8b5bd9-2ee3-4d91-beb9-c6799a5db3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:55.159976 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:55.239711 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 719/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:02:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:02:55.276979 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970eebf-a9a4-4e06-8b75-fc5db0c06f5b None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:02:55.278534 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-e970eebf-a9a4-4e06-8b75-fc5db0c06f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:55.383324 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970eebf-a9a4-4e06-8b75-fc5db0c06f5b None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:02:55.449032 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:55.456321 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61028855-220e-4648-b7e6-eed58db8c2d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:02:55.457025 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61028855-220e-4648-b7e6-eed58db8c2d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:55.457308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61028855-220e-4648-b7e6-eed58db8c2d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:55.457588 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61028855-220e-4648-b7e6-eed58db8c2d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:55.459212 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-61028855-220e-4648-b7e6-eed58db8c2d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:55.566884 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61028855-220e-4648-b7e6-eed58db8c2d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:02:55.568639 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61028855-220e-4648-b7e6-eed58db8c2d7 service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:02:55.587296 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 748/1467] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:02:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:02:57.664334 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160283f-53f5-45c8-9c2d-f441238c91b4 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:57.664926 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160283f-53f5-45c8-9c2d-f441238c91b4 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:57.664926 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160283f-53f5-45c8-9c2d-f441238c91b4 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:57.666253 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 720/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:02:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:02:57.671435 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddbb825-f78c-428e-84fe-5c873d710cbd None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:57.671847 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddbb825-f78c-428e-84fe-5c873d710cbd None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:57.671993 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddbb825-f78c-428e-84fe-5c873d710cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:57.737552 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-1ddbb825-f78c-428e-84fe-5c873d710cbd None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:02:57.738789 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-1ddbb825-f78c-428e-84fe-5c873d710cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:57.788340 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:57.837393 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 749/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:02:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:02:57.873601 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5160283f-53f5-45c8-9c2d-f441238c91b4 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:02:57.875116 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5160283f-53f5-45c8-9c2d-f441238c91b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:57.978709 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5160283f-53f5-45c8-9c2d-f441238c91b4 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:02:58.025955 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:02:58.051429 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9190f3-4d40-48a9-a510-96af14382ac0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:02:58.052047 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9190f3-4d40-48a9-a510-96af14382ac0 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:02:58.052327 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9190f3-4d40-48a9-a510-96af14382ac0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:02:58.052632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9190f3-4d40-48a9-a510-96af14382ac0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:02:58.054769 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3a9190f3-4d40-48a9-a510-96af14382ac0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:02:58.162469 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9190f3-4d40-48a9-a510-96af14382ac0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:02:58.163928 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9190f3-4d40-48a9-a510-96af14382ac0 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:02:58.181916 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 721/1470] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:02:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:03:07.835004 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=22 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:03:08.179661 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=66 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:03:17.740440 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f6827-ce1e-41ee-aa18-f2ff57fc7e43 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:03:17.740989 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f6827-ce1e-41ee-aa18-f2ff57fc7e43 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:03:17.741093 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f6827-ce1e-41ee-aa18-f2ff57fc7e43 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:03:17.742439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 750/1471] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 14:03:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:03:17.747640 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fbfd8-1d9c-490e-aabd-c5587307214a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:03:17.748063 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fbfd8-1d9c-490e-aabd-c5587307214a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:03:17.748191 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fbfd8-1d9c-490e-aabd-c5587307214a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:03:17.830326 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-616fbfd8-1d9c-490e-aabd-c5587307214a None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:03:17.831538 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-616fbfd8-1d9c-490e-aabd-c5587307214a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:03:17.935792 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 722/1472] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 14:03:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:03:18.023182 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c1fe5-037d-4896-bada-d3cafc6a9fef None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:03:18.023472 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c1fe5-037d-4896-bada-d3cafc6a9fef None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:03:18.023757 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c1fe5-037d-4896-bada-d3cafc6a9fef None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:03:18.025132 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 751/1473] 10.222.0.48 () {54 vars in 966 bytes} [Mon Aug 1 14:03:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:03:18.030019 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba006f15-cef7-42aa-b318-a14162787678 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:03:18.030310 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba006f15-cef7-42aa-b318-a14162787678 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:03:18.030591 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba006f15-cef7-42aa-b318-a14162787678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:03:18.106025 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ba006f15-cef7-42aa-b318-a14162787678 None None] MFA Rules not processed for user `39a66f7362964389be039937a786bb0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:03:18.107225 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ba006f15-cef7-42aa-b318-a14162787678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:03:18.127656 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:03:18.183098 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 723/1474] 10.222.0.48 () {58 vars in 1063 bytes} [Mon Aug 1 14:03:18 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:03:18.190636 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c1fe5-037d-4896-bada-d3cafc6a9fef None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:03:18.192113 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-289c1fe5-037d-4896-bada-d3cafc6a9fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:03:18.293648 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c1fe5-037d-4896-bada-d3cafc6a9fef None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:03:18.366193 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa634ec-55e0-47e0-ba00-fde8864a7715 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39a66f7362964389be039937a786bb0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:03:18.366808 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa634ec-55e0-47e0-ba00-fde8864a7715 service glance] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:03:18.367090 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa634ec-55e0-47e0-ba00-fde8864a7715 service glance] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:03:18.367368 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa634ec-55e0-47e0-ba00-fde8864a7715 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:03:18.369019 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-2aa634ec-55e0-47e0-ba00-fde8864a7715 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:03:18.421679 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:03:18.476539 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa634ec-55e0-47e0-ba00-fde8864a7715 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:03:18.477978 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa634ec-55e0-47e0-ba00-fde8864a7715 service glance] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:03:18.496850 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 752/1475] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:03:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:03:20.546315 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7e854-70cd-4af8-ac57-98a3f7bfe692 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:03:20.546878 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7e854-70cd-4af8-ac57-98a3f7bfe692 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:03:20.546878 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7e854-70cd-4af8-ac57-98a3f7bfe692 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:03:20.548204 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 724/1476] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 14:03:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:03:20.553359 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0084f-5c02-4eef-bd75-065fb2e1cbdc None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:03:20.553809 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0084f-5c02-4eef-bd75-065fb2e1cbdc None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:03:20.553915 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0084f-5c02-4eef-bd75-065fb2e1cbdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:03:20.626949 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-1ad0084f-5c02-4eef-bd75-065fb2e1cbdc None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:03:20.628176 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-1ad0084f-5c02-4eef-bd75-065fb2e1cbdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:03:20.724880 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 753/1477] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 14:03:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:03:20.747691 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7e854-70cd-4af8-ac57-98a3f7bfe692 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:03:20.749263 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-42c7e854-70cd-4af8-ac57-98a3f7bfe692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:03:20.871117 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7e854-70cd-4af8-ac57-98a3f7bfe692 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:03:20.919448 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:03:20.965626 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801bee16-4ddf-491c-a5d3-689bd7d34c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:03:20.966257 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801bee16-4ddf-491c-a5d3-689bd7d34c53 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:03:20.967639 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801bee16-4ddf-491c-a5d3-689bd7d34c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:03:20.967639 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801bee16-4ddf-491c-a5d3-689bd7d34c53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:03:20.967639 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801bee16-4ddf-491c-a5d3-689bd7d34c53 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:03:20.969315 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: WARNING py.warnings [None req-801bee16-4ddf-491c-a5d3-689bd7d34c53 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:03:20.969315 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: warnings.warn(msg) Aug 01 14:03:20.969315 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: Aug 01 14:03:20.970455 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801bee16-4ddf-491c-a5d3-689bd7d34c53 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:03:20.992375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 725/1478] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 14:03:20 2022] GET /identity/v3/projects => generated 2553 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:03:23.009292 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5a242-87fd-435e-91d0-bf5479e28ea9 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:03:23.009912 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5a242-87fd-435e-91d0-bf5479e28ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:03:23.009912 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5a242-87fd-435e-91d0-bf5479e28ea9 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:03:23.011190 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 754/1479] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 14:03:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:03:23.016773 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e4ab0-41b4-4c9a-b6ee-99f1de2a811a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:03:23.017222 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e4ab0-41b4-4c9a-b6ee-99f1de2a811a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:03:23.017328 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e4ab0-41b4-4c9a-b6ee-99f1de2a811a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:03:23.083327 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-2f0e4ab0-41b4-4c9a-b6ee-99f1de2a811a None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:03:23.084454 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2f0e4ab0-41b4-4c9a-b6ee-99f1de2a811a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:03:23.182067 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 726/1480] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 14:03:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:03:23.225372 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5a242-87fd-435e-91d0-bf5479e28ea9 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:03:23.226986 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-e5a5a242-87fd-435e-91d0-bf5479e28ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:03:23.237110 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:03:23.379149 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5a242-87fd-435e-91d0-bf5479e28ea9 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:03:23.457083 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:03:23.490948 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc57ca-b8ad-48f3-a222-80785b9fc8de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:03:23.491562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc57ca-b8ad-48f3-a222-80785b9fc8de service nova] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:03:23.491847 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc57ca-b8ad-48f3-a222-80785b9fc8de service nova] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:03:23.492119 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc57ca-b8ad-48f3-a222-80785b9fc8de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:03:23.493766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-5fdc57ca-b8ad-48f3-a222-80785b9fc8de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:03:23.581697 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdc57ca-b8ad-48f3-a222-80785b9fc8de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:03:23.583788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdc57ca-b8ad-48f3-a222-80785b9fc8de service nova] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:03:23.601937 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 755/1481] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:03:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:03:25.774353 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731f467-0482-4a56-b8f2-59b7864ca5ed None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:03:25.774916 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731f467-0482-4a56-b8f2-59b7864ca5ed None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:03:25.775038 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731f467-0482-4a56-b8f2-59b7864ca5ed None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:03:25.776429 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 727/1482] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 14:03:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:03:25.781660 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf2c9f-6c1a-4a5b-b566-ca93ef8febc4 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:03:25.782074 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf2c9f-6c1a-4a5b-b566-ca93ef8febc4 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:03:25.782214 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf2c9f-6c1a-4a5b-b566-ca93ef8febc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:03:25.847924 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-83bf2c9f-6c1a-4a5b-b566-ca93ef8febc4 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:03:25.849774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-83bf2c9f-6c1a-4a5b-b566-ca93ef8febc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:03:25.895373 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:03:25.945348 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 756/1483] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 14:03:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:03:25.981515 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731f467-0482-4a56-b8f2-59b7864ca5ed None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:03:25.983075 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8731f467-0482-4a56-b8f2-59b7864ca5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:03:26.006242 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:03:26.085922 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731f467-0482-4a56-b8f2-59b7864ca5ed None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:03:26.159399 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831bcdc6-5fca-4c8d-97ca-7eb4fd2d31fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:03:26.160008 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bcdc6-5fca-4c8d-97ca-7eb4fd2d31fc service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:03:26.160284 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bcdc6-5fca-4c8d-97ca-7eb4fd2d31fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:03:26.160638 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bcdc6-5fca-4c8d-97ca-7eb4fd2d31fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:03:26.162237 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-831bcdc6-5fca-4c8d-97ca-7eb4fd2d31fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:03:26.234493 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:03:26.270558 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831bcdc6-5fca-4c8d-97ca-7eb4fd2d31fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:03:26.272009 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831bcdc6-5fca-4c8d-97ca-7eb4fd2d31fc service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:03:26.290189 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 728/1484] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:03:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:03:28.362200 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a85ad4-a453-45cd-8b43-c779865789b3 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:03:28.362762 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a85ad4-a453-45cd-8b43-c779865789b3 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:03:28.362762 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a85ad4-a453-45cd-8b43-c779865789b3 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:03:28.364692 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 757/1485] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 14:03:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:03:28.369916 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828c086-2a36-4743-8c6b-b0245129a33b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:03:28.370357 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828c086-2a36-4743-8c6b-b0245129a33b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:03:28.370471 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828c086-2a36-4743-8c6b-b0245129a33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:03:28.436444 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d828c086-2a36-4743-8c6b-b0245129a33b None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:03:28.437692 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d828c086-2a36-4743-8c6b-b0245129a33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:03:28.535184 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 729/1486] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 14:03:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:03:28.573510 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a85ad4-a453-45cd-8b43-c779865789b3 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:03:28.575081 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-c8a85ad4-a453-45cd-8b43-c779865789b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:03:28.680884 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a85ad4-a453-45cd-8b43-c779865789b3 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:03:28.759358 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:03:28.761600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5215b220-3f3a-4882-ade0-95cd3a593b9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:03:28.762506 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215b220-3f3a-4882-ade0-95cd3a593b9d service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:03:28.762974 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215b220-3f3a-4882-ade0-95cd3a593b9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:03:28.763481 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215b220-3f3a-4882-ade0-95cd3a593b9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:03:28.765341 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-5215b220-3f3a-4882-ade0-95cd3a593b9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:03:28.874706 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5215b220-3f3a-4882-ade0-95cd3a593b9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:03:28.876187 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5215b220-3f3a-4882-ade0-95cd3a593b9d service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:03:28.894713 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 758/1487] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:03:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:03:31.045315 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71cb8cc-fafc-4575-8cd8-37bb1ee28b9b None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:03:31.045873 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71cb8cc-fafc-4575-8cd8-37bb1ee28b9b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:03:31.045873 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71cb8cc-fafc-4575-8cd8-37bb1ee28b9b None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:03:31.047242 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 730/1488] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 14:03:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:03:31.052778 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81f57c-a316-4801-95d1-2835c00e47fe None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:03:31.053170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81f57c-a316-4801-95d1-2835c00e47fe None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:03:31.053343 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81f57c-a316-4801-95d1-2835c00e47fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:03:31.121328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-5d81f57c-a316-4801-95d1-2835c00e47fe None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:03:31.122545 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-5d81f57c-a316-4801-95d1-2835c00e47fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:03:31.174947 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:03:31.219807 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 759/1489] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 14:03:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:03:31.260191 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71cb8cc-fafc-4575-8cd8-37bb1ee28b9b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:03:31.261773 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e71cb8cc-fafc-4575-8cd8-37bb1ee28b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:03:31.298115 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:03:31.376129 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71cb8cc-fafc-4575-8cd8-37bb1ee28b9b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:03:31.450089 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1afc66-18fa-4f9c-bfe8-4a8d08844356 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:03:31.450710 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1afc66-18fa-4f9c-bfe8-4a8d08844356 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:03:31.451002 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1afc66-18fa-4f9c-bfe8-4a8d08844356 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:03:31.451276 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1afc66-18fa-4f9c-bfe8-4a8d08844356 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:03:31.452925 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-bf1afc66-18fa-4f9c-bfe8-4a8d08844356 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:03:31.533341 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:03:31.561965 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1afc66-18fa-4f9c-bfe8-4a8d08844356 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:03:31.563338 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1afc66-18fa-4f9c-bfe8-4a8d08844356 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:03:31.581383 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 731/1490] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:03:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:03:33.790670 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2c5a5-d2bc-4dda-9782-bce83dc11ce5 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:03:33.791210 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2c5a5-d2bc-4dda-9782-bce83dc11ce5 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:03:33.791210 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2c5a5-d2bc-4dda-9782-bce83dc11ce5 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:03:33.792593 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 760/1491] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 14:03:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:03:33.797837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cce9c-0faa-4464-bf11-3c9b9bdb0595 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:03:33.798283 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cce9c-0faa-4464-bf11-3c9b9bdb0595 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:03:33.798470 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cce9c-0faa-4464-bf11-3c9b9bdb0595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:03:33.865861 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a12cce9c-0faa-4464-bf11-3c9b9bdb0595 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:03:33.867082 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a12cce9c-0faa-4464-bf11-3c9b9bdb0595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:03:33.968218 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 732/1492] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 14:03:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:03:34.008386 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b2c5a5-d2bc-4dda-9782-bce83dc11ce5 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:03:34.010074 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-03b2c5a5-d2bc-4dda-9782-bce83dc11ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:03:34.114036 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b2c5a5-d2bc-4dda-9782-bce83dc11ce5 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:03:34.186336 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475e975-b7e2-461d-a4ab-923143695cd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:03:34.186949 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475e975-b7e2-461d-a4ab-923143695cd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:03:34.187235 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475e975-b7e2-461d-a4ab-923143695cd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:03:34.187513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475e975-b7e2-461d-a4ab-923143695cd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:03:34.189162 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-2475e975-b7e2-461d-a4ab-923143695cd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:03:34.198105 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:03:34.297036 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2475e975-b7e2-461d-a4ab-923143695cd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:03:34.298393 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2475e975-b7e2-461d-a4ab-923143695cd2 service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:03:34.315942 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 761/1493] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:03:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:03:43.966430 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=82 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:03:44.313773 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=50 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:02.312049 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795ae6f-320e-4066-8435-62d1e1e6b7f4 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:02.312680 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795ae6f-320e-4066-8435-62d1e1e6b7f4 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:02.312680 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795ae6f-320e-4066-8435-62d1e1e6b7f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:02.387733 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-9795ae6f-320e-4066-8435-62d1e1e6b7f4 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:02.388907 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9795ae6f-320e-4066-8435-62d1e1e6b7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:02.485621 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 733/1494] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:05:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:05:02.493579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e47d83-6acb-489b-b59e-4f4fd9ed6817 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:02.494055 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e47d83-6acb-489b-b59e-4f4fd9ed6817 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:02.494161 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e47d83-6acb-489b-b59e-4f4fd9ed6817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:02.569386 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-39e47d83-6acb-489b-b59e-4f4fd9ed6817 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:02.570597 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-39e47d83-6acb-489b-b59e-4f4fd9ed6817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:02.665802 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 762/1495] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:05:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:05:02.670933 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795ae6f-320e-4066-8435-62d1e1e6b7f4 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:02.672112 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9795ae6f-320e-4066-8435-62d1e1e6b7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:02.683225 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795ae6f-320e-4066-8435-62d1e1e6b7f4 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:02.744317 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:02.778771 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e0462-0258-4e8c-8418-80e0d59ec487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:02.779391 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e0462-0258-4e8c-8418-80e0d59ec487 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:02.779677 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e0462-0258-4e8c-8418-80e0d59ec487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:02.779957 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e0462-0258-4e8c-8418-80e0d59ec487 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:02.780692 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3e0462-0258-4e8c-8418-80e0d59ec487 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:05:02.781002 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3e0462-0258-4e8c-8418-80e0d59ec487 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:02.782005 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: WARNING py.warnings [None req-4d3e0462-0258-4e8c-8418-80e0d59ec487 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:05:02.782005 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: warnings.warn(msg) Aug 01 14:05:02.782005 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: Aug 01 14:05:02.782891 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3e0462-0258-4e8c-8418-80e0d59ec487 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:02.794367 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 734/1496] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:05:02.802954 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e47d83-6acb-489b-b59e-4f4fd9ed6817 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:02.812755 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e47d83-6acb-489b-b59e-4f4fd9ed6817 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:02.873243 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:02.906290 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1353292d-3c5b-40d2-a668-d3d9b48e246b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:02.906909 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353292d-3c5b-40d2-a668-d3d9b48e246b admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:02.907186 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353292d-3c5b-40d2-a668-d3d9b48e246b admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:02.907458 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353292d-3c5b-40d2-a668-d3d9b48e246b admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:02.908108 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1353292d-3c5b-40d2-a668-d3d9b48e246b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:02.909851 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: WARNING py.warnings [None req-1353292d-3c5b-40d2-a668-d3d9b48e246b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:05:02.909851 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: warnings.warn(msg) Aug 01 14:05:02.909851 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: Aug 01 14:05:02.911077 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1353292d-3c5b-40d2-a668-d3d9b48e246b admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:02.966576 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 763/1497] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:05:02 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:05:02.970896 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e0462-0258-4e8c-8418-80e0d59ec487 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:02.972412 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4d3e0462-0258-4e8c-8418-80e0d59ec487 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:03.005920 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e0462-0258-4e8c-8418-80e0d59ec487 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:03.102464 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae7f037-0671-451e-ba8c-f33832dc2a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:03.103079 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae7f037-0671-451e-ba8c-f33832dc2a79 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:03.103360 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae7f037-0671-451e-ba8c-f33832dc2a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:03.103631 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae7f037-0671-451e-ba8c-f33832dc2a79 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:03.104282 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae7f037-0671-451e-ba8c-f33832dc2a79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:03.106217 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: WARNING py.warnings [None req-2ae7f037-0671-451e-ba8c-f33832dc2a79 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:05:03.106217 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: warnings.warn(msg) Aug 01 14:05:03.106217 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: Aug 01 14:05:03.107399 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae7f037-0671-451e-ba8c-f33832dc2a79 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:03.147894 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 735/1498] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:05:02 2022] POST /identity/v3/users => generated 289 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:05:03.152708 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1353292d-3c5b-40d2-a668-d3d9b48e246b admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:03.154222 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-1353292d-3c5b-40d2-a668-d3d9b48e246b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:03.214463 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1353292d-3c5b-40d2-a668-d3d9b48e246b admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:03.296715 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:03.308282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880da225-7216-4ce4-b3ea-423b348be7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:03.309322 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880da225-7216-4ce4-b3ea-423b348be7bb admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:03.309608 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880da225-7216-4ce4-b3ea-423b348be7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:03.309888 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880da225-7216-4ce4-b3ea-423b348be7bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:03.310590 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880da225-7216-4ce4-b3ea-423b348be7bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:03.311608 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: WARNING py.warnings [None req-880da225-7216-4ce4-b3ea-423b348be7bb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:05:03.311608 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: warnings.warn(msg) Aug 01 14:05:03.311608 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: Aug 01 14:05:03.312519 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880da225-7216-4ce4-b3ea-423b348be7bb admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:03.325264 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 764/1499] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:05:03.329264 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae7f037-0671-451e-ba8c-f33832dc2a79 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:03.330761 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2ae7f037-0671-451e-ba8c-f33832dc2a79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:03.342734 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae7f037-0671-451e-ba8c-f33832dc2a79 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:03.368785 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:03.438884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6e91e7-7b0d-4588-850e-6d31df8bde50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:03.439501 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e91e7-7b0d-4588-850e-6d31df8bde50 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:03.439787 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e91e7-7b0d-4588-850e-6d31df8bde50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:03.440061 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e91e7-7b0d-4588-850e-6d31df8bde50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:03.440796 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6e91e7-7b0d-4588-850e-6d31df8bde50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:03.441910 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: WARNING py.warnings [None req-af6e91e7-7b0d-4588-850e-6d31df8bde50 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:05:03.441910 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: warnings.warn(msg) Aug 01 14:05:03.441910 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: Aug 01 14:05:03.442860 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6e91e7-7b0d-4588-850e-6d31df8bde50 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:03.455614 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 736/1500] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:05:03.459843 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880da225-7216-4ce4-b3ea-423b348be7bb admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:03.461432 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-880da225-7216-4ce4-b3ea-423b348be7bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:03.494631 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880da225-7216-4ce4-b3ea-423b348be7bb admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:03.587629 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b2c7bb-49c9-4e7c-92a1-039da643234b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:03.588217 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b2c7bb-49c9-4e7c-92a1-039da643234b admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:03.588518 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b2c7bb-49c9-4e7c-92a1-039da643234b admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:03.588798 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b2c7bb-49c9-4e7c-92a1-039da643234b admin admin] PATH_INFO: `/v3/projects/cc8c79374ef846709233f00c2baeca5b/users/94defe5973b649d79ba15507bcafb80a/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:03.610731 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b2c7bb-49c9-4e7c-92a1-039da643234b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc8c79374ef846709233f00c2baeca5b, user_id=94defe5973b649d79ba15507bcafb80a, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:03.611746 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: WARNING py.warnings [None req-89b2c7bb-49c9-4e7c-92a1-039da643234b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:05:03.611746 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: warnings.warn(msg) Aug 01 14:05:03.611746 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: Aug 01 14:05:03.613706 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b2c7bb-49c9-4e7c-92a1-039da643234b admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:03.638168 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 765/1501] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:05:03 2022] PUT /identity/v3/projects/cc8c79374ef846709233f00c2baeca5b/users/94defe5973b649d79ba15507bcafb80a/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:05:03.650255 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2683ff97-d711-435b-8c90-a4da4f37a9b9 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:03.650549 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2683ff97-d711-435b-8c90-a4da4f37a9b9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:03.650834 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2683ff97-d711-435b-8c90-a4da4f37a9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:03.710967 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-2683ff97-d711-435b-8c90-a4da4f37a9b9 None None] MFA Rules not processed for user `94defe5973b649d79ba15507bcafb80a`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:03.712187 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2683ff97-d711-435b-8c90-a4da4f37a9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:03.791238 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 737/1502] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:05:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:05:03.804466 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1949308-d3b4-45b1-8916-92ca7a8e4194 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:03.804789 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1949308-d3b4-45b1-8916-92ca7a8e4194 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:03.805077 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1949308-d3b4-45b1-8916-92ca7a8e4194 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:03.806437 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 766/1503] 10.222.0.48 () {54 vars in 964 bytes} [Mon Aug 1 14:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:03.811738 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc15cb-d0c0-4555-952e-92f4d3ced7a3 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:03.812028 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc15cb-d0c0-4555-952e-92f4d3ced7a3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:03.812300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc15cb-d0c0-4555-952e-92f4d3ced7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:03.844363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:03.888784 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-44fc15cb-d0c0-4555-952e-92f4d3ced7a3 None None] MFA Rules not processed for user `b185b052f9df4d3f9847d8f9b4cfda5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:03.889982 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-44fc15cb-d0c0-4555-952e-92f4d3ced7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:03.966432 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 738/1504] 10.222.0.48 () {58 vars in 1061 bytes} [Mon Aug 1 14:05:03 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:05:03.974223 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1949308-d3b4-45b1-8916-92ca7a8e4194 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:03.975772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-c1949308-d3b4-45b1-8916-92ca7a8e4194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:04.026034 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:04.081490 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1949308-d3b4-45b1-8916-92ca7a8e4194 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:04.157740 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75595cfa-2a13-4365-bdb4-509b723cc63b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:04.158366 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75595cfa-2a13-4365-bdb4-509b723cc63b service cinder] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:04.158644 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75595cfa-2a13-4365-bdb4-509b723cc63b service cinder] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:04.158914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75595cfa-2a13-4365-bdb4-509b723cc63b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:04.160535 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-75595cfa-2a13-4365-bdb4-509b723cc63b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:04.262569 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75595cfa-2a13-4365-bdb4-509b723cc63b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:04.263989 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75595cfa-2a13-4365-bdb4-509b723cc63b service cinder] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:04.267864 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:04.282846 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 767/1505] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:05:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:05:04.327552 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aeb367-0c59-4551-b46f-d4e229764960 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:04.327842 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aeb367-0c59-4551-b46f-d4e229764960 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:04.328118 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aeb367-0c59-4551-b46f-d4e229764960 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:04.329403 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 739/1506] 10.222.0.48 () {52 vars in 830 bytes} [Mon Aug 1 14:05:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:05:04.332972 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75595cfa-2a13-4365-bdb4-509b723cc63b service cinder] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:04.334167 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-75595cfa-2a13-4365-bdb4-509b723cc63b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:04.439404 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75595cfa-2a13-4365-bdb4-509b723cc63b service cinder] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:04.531869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b94b9-7475-4a34-8e6c-4b173f9a23b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:04.532637 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b94b9-7475-4a34-8e6c-4b173f9a23b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:04.532832 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b94b9-7475-4a34-8e6c-4b173f9a23b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:04.533121 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b94b9-7475-4a34-8e6c-4b173f9a23b5 admin admin] PATH_INFO: `/v3/users/94defe5973b649d79ba15507bcafb80a` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:04.545464 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2b94b9-7475-4a34-8e6c-4b173f9a23b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94defe5973b649d79ba15507bcafb80a)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:04.546487 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: WARNING py.warnings [None req-1c2b94b9-7475-4a34-8e6c-4b173f9a23b5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:05:04.546487 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: warnings.warn(msg) Aug 01 14:05:04.546487 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: Aug 01 14:05:04.547700 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2b94b9-7475-4a34-8e6c-4b173f9a23b5 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:04.557271 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:04.614753 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-1c2b94b9-7475-4a34-8e6c-4b173f9a23b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94defe5973b649d79ba15507bcafb80a'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:05:04.615319 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-1c2b94b9-7475-4a34-8e6c-4b173f9a23b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94defe5973b649d79ba15507bcafb80a'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:05:04.638675 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-1c2b94b9-7475-4a34-8e6c-4b173f9a23b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94defe5973b649d79ba15507bcafb80a'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:05:04.647614 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-1c2b94b9-7475-4a34-8e6c-4b173f9a23b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94defe5973b649d79ba15507bcafb80a'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:05:04.648078 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-1c2b94b9-7475-4a34-8e6c-4b173f9a23b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94defe5973b649d79ba15507bcafb80a'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:05:04.667071 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 768/1507] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:05:04 2022] DELETE /identity/v3/users/94defe5973b649d79ba15507bcafb80a => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:05:04.671503 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58cf433-2fbd-4914-8ae0-42449845b1f2 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:04.671794 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58cf433-2fbd-4914-8ae0-42449845b1f2 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:04.672072 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58cf433-2fbd-4914-8ae0-42449845b1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:04.728939 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b58cf433-2fbd-4914-8ae0-42449845b1f2 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:05:04.730087 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b58cf433-2fbd-4914-8ae0-42449845b1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:04.813259 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:04.826396 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 740/1508] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:05:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:05:04.836663 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b94b9-7475-4a34-8e6c-4b173f9a23b5 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:04.837903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-1c2b94b9-7475-4a34-8e6c-4b173f9a23b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:04.938287 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b94b9-7475-4a34-8e6c-4b173f9a23b5 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:04.992010 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:05.013116 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d7026-02a4-4bbd-982f-5f692ce7d533 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:05.013771 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d7026-02a4-4bbd-982f-5f692ce7d533 service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:05.014052 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d7026-02a4-4bbd-982f-5f692ce7d533 service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:05.014322 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d7026-02a4-4bbd-982f-5f692ce7d533 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:05.015981 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-1c3d7026-02a4-4bbd-982f-5f692ce7d533 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:05.124279 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d7026-02a4-4bbd-982f-5f692ce7d533 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:05.125836 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d7026-02a4-4bbd-982f-5f692ce7d533 service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:05.144632 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 769/1509] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:05:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:05:05.481246 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58cf433-2fbd-4914-8ae0-42449845b1f2 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:05:05.482485 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b58cf433-2fbd-4914-8ae0-42449845b1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:05:05.493955 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58cf433-2fbd-4914-8ae0-42449845b1f2 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:05:05.592172 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2383b7-4266-46df-916c-8260d178222a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:05:05.592875 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2383b7-4266-46df-916c-8260d178222a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:05:05.593163 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2383b7-4266-46df-916c-8260d178222a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:05:05.593441 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2383b7-4266-46df-916c-8260d178222a admin admin] PATH_INFO: `/v3/projects/cc8c79374ef846709233f00c2baeca5b` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:05:05.604201 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2383b7-4266-46df-916c-8260d178222a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc8c79374ef846709233f00c2baeca5b)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:05:05.605333 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: WARNING py.warnings [None req-ae2383b7-4266-46df-916c-8260d178222a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:05:05.605333 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: warnings.warn(msg) Aug 01 14:05:05.605333 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: Aug 01 14:05:05.606882 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2383b7-4266-46df-916c-8260d178222a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:05:05.666354 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ae2383b7-4266-46df-916c-8260d178222a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc8c79374ef846709233f00c2baeca5b', 'request_id': 'req-ae2383b7-4266-46df-916c-8260d178222a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:05:05.675894 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ae2383b7-4266-46df-916c-8260d178222a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc8c79374ef846709233f00c2baeca5b', 'request_id': 'req-ae2383b7-4266-46df-916c-8260d178222a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:05:05.690319 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ae2383b7-4266-46df-916c-8260d178222a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc8c79374ef846709233f00c2baeca5b', 'request_id': 'req-ae2383b7-4266-46df-916c-8260d178222a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:05:05.714310 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ae2383b7-4266-46df-916c-8260d178222a admin admin] The token cache is being invalidate because project cc8c79374ef846709233f00c2baeca5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:05:05.715166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ae2383b7-4266-46df-916c-8260d178222a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:05:05.715905 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ae2383b7-4266-46df-916c-8260d178222a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:05:05.717088 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 741/1510] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:05:05 2022] DELETE /identity/v3/projects/cc8c79374ef846709233f00c2baeca5b => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:05:15.142282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=62 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:05:15.712216 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:48.969259 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b4b92a-1761-4013-b5a8-126287f9e0df None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:48.969840 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b4b92a-1761-4013-b5a8-126287f9e0df None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:48.969840 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b4b92a-1761-4013-b5a8-126287f9e0df None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:48.971168 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 770/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 14:06:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:06:48.976553 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ae1ab-a71d-4f71-83f0-3c391683ae54 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:48.977004 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ae1ab-a71d-4f71-83f0-3c391683ae54 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:48.977153 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ae1ab-a71d-4f71-83f0-3c391683ae54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:49.057566 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4f0ae1ab-a71d-4f71-83f0-3c391683ae54 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:49.058807 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4f0ae1ab-a71d-4f71-83f0-3c391683ae54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:49.165555 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 742/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 14:06:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:06:49.316757 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b4b92a-1761-4013-b5a8-126287f9e0df None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:49.318977 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-06b4b92a-1761-4013-b5a8-126287f9e0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:49.439837 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b4b92a-1761-4013-b5a8-126287f9e0df None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:49.445570 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368a4675-017e-4533-8e83-589732f8154e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:49.446296 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a4675-017e-4533-8e83-589732f8154e service nova] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:49.446584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a4675-017e-4533-8e83-589732f8154e service nova] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:49.446866 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a4675-017e-4533-8e83-589732f8154e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:49.455694 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368a4675-017e-4533-8e83-589732f8154e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:49.457321 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368a4675-017e-4533-8e83-589732f8154e service nova] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:49.466712 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 771/1513] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:06:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:06:50.289340 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa6af8-7a50-498b-9586-a4883d32885a None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:50.289915 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa6af8-7a50-498b-9586-a4883d32885a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:50.289915 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa6af8-7a50-498b-9586-a4883d32885a None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:50.291263 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 743/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 14:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:06:50.296714 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb436c-86b2-43b3-bd48-8db086c7efe7 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:50.297094 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb436c-86b2-43b3-bd48-8db086c7efe7 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:50.297287 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb436c-86b2-43b3-bd48-8db086c7efe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:50.316406 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-97cb436c-86b2-43b3-bd48-8db086c7efe7 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:50.317636 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-97cb436c-86b2-43b3-bd48-8db086c7efe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:50.325965 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 772/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 14:06:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:06:50.478208 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa6af8-7a50-498b-9586-a4883d32885a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:06:50.492855 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa6af8-7a50-498b-9586-a4883d32885a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:06:50.498905 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b17cd0a-80f3-4fa7-bb5e-6220450c9679 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:06:50.499519 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17cd0a-80f3-4fa7-bb5e-6220450c9679 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:50.499807 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17cd0a-80f3-4fa7-bb5e-6220450c9679 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:50.500090 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17cd0a-80f3-4fa7-bb5e-6220450c9679 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:50.508112 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b17cd0a-80f3-4fa7-bb5e-6220450c9679 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:06:50.509725 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b17cd0a-80f3-4fa7-bb5e-6220450c9679 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:06:50.519035 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 744/1516] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:06:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:06:50.930946 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b12d31-ea20-4cb6-8ad3-a82ec6be5baa None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:50.931239 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b12d31-ea20-4cb6-8ad3-a82ec6be5baa None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:50.931527 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b12d31-ea20-4cb6-8ad3-a82ec6be5baa None None] PATH_INFO: `/v3` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:50.932902 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 773/1517] 10.222.0.71 () {54 vars in 925 bytes} [Mon Aug 1 14:06:50 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:06:50.938945 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9c139-8c1e-481a-90cd-c8db483394b1 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:06:50.939261 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9c139-8c1e-481a-90cd-c8db483394b1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:06:50.939538 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9c139-8c1e-481a-90cd-c8db483394b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:06:50.998688 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-01a9c139-8c1e-481a-90cd-c8db483394b1 None None] MFA Rules not processed for user `e294fe796a1a4ac89041a532856ad627`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:06:50.999889 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-01a9c139-8c1e-481a-90cd-c8db483394b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:06:51.016153 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:06:51.070363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 745/1518] 10.222.0.71 () {58 vars in 1015 bytes} [Mon Aug 1 14:06:50 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:07:00.311727 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=56 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:00.657733 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d41cd-fae4-4c6e-a2a8-24b797369956 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:00.658055 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d41cd-fae4-4c6e-a2a8-24b797369956 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:00.658344 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d41cd-fae4-4c6e-a2a8-24b797369956 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:00.659696 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 774/1519] 10.222.0.48 () {54 vars in 961 bytes} [Mon Aug 1 14:07:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:00.664744 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684685c-6dea-4076-a09f-4622f564d287 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:00.665217 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684685c-6dea-4076-a09f-4622f564d287 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:00.665332 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684685c-6dea-4076-a09f-4622f564d287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:00.686944 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f684685c-6dea-4076-a09f-4622f564d287 None None] MFA Rules not processed for user `e294fe796a1a4ac89041a532856ad627`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:00.688162 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f684685c-6dea-4076-a09f-4622f564d287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:00.696946 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 746/1520] 10.222.0.48 () {58 vars in 1058 bytes} [Mon Aug 1 14:07:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:07:00.705145 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058d41cd-fae4-4c6e-a2a8-24b797369956 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:00.719451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058d41cd-fae4-4c6e-a2a8-24b797369956 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:00.724495 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8829295c-074b-4dbd-a4a1-1ae583094e55 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e294fe796a1a4ac89041a532856ad627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:00.725193 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829295c-074b-4dbd-a4a1-1ae583094e55 service placement] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:00.725737 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829295c-074b-4dbd-a4a1-1ae583094e55 service placement] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:00.726018 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829295c-074b-4dbd-a4a1-1ae583094e55 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:00.735521 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8829295c-074b-4dbd-a4a1-1ae583094e55 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:00.737338 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8829295c-074b-4dbd-a4a1-1ae583094e55 service placement] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:00.747613 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 775/1521] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:07:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:07:01.739424 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea03e39-466f-49e2-a58c-164226fedaf2 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:01.739988 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea03e39-466f-49e2-a58c-164226fedaf2 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:01.740098 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea03e39-466f-49e2-a58c-164226fedaf2 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:01.741420 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 747/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 14:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:01.746087 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7c800-d4ec-4fad-84c1-73028e707add None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:01.746523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7c800-d4ec-4fad-84c1-73028e707add None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:01.746686 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7c800-d4ec-4fad-84c1-73028e707add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:01.765979 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-5cc7c800-d4ec-4fad-84c1-73028e707add None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:01.767186 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-5cc7c800-d4ec-4fad-84c1-73028e707add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:01.775103 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 776/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 14:07:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:07:01.928213 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea03e39-466f-49e2-a58c-164226fedaf2 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:01.942423 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea03e39-466f-49e2-a58c-164226fedaf2 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:01.947684 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca34f11-153a-4ba7-9349-c88193ba2661 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:01.948287 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca34f11-153a-4ba7-9349-c88193ba2661 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:01.948597 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca34f11-153a-4ba7-9349-c88193ba2661 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:01.948874 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca34f11-153a-4ba7-9349-c88193ba2661 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:01.957296 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca34f11-153a-4ba7-9349-c88193ba2661 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:01.958860 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca34f11-153a-4ba7-9349-c88193ba2661 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:01.968613 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 748/1524] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:07:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:07:02.791800 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca3ec4a-6b42-4eae-888a-9938e7e8006d None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:02.792360 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca3ec4a-6b42-4eae-888a-9938e7e8006d None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:02.792360 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca3ec4a-6b42-4eae-888a-9938e7e8006d None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:02.794362 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 777/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 14:07:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:02.799378 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74530651-f2e8-402e-b16a-089b4f4274d0 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:02.799864 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74530651-f2e8-402e-b16a-089b4f4274d0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:02.800057 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74530651-f2e8-402e-b16a-089b4f4274d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:02.820465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-74530651-f2e8-402e-b16a-089b4f4274d0 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:02.821708 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-74530651-f2e8-402e-b16a-089b4f4274d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:02.829666 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 749/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 14:07:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:07:02.981752 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca3ec4a-6b42-4eae-888a-9938e7e8006d None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:02.995931 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca3ec4a-6b42-4eae-888a-9938e7e8006d None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:03.001922 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74f317-1f5e-4557-a82e-23d8f20a6818 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:03.002526 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74f317-1f5e-4557-a82e-23d8f20a6818 service nova] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:03.002841 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74f317-1f5e-4557-a82e-23d8f20a6818 service nova] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:03.003118 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74f317-1f5e-4557-a82e-23d8f20a6818 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:03.011099 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a74f317-1f5e-4557-a82e-23d8f20a6818 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:03.012702 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a74f317-1f5e-4557-a82e-23d8f20a6818 service nova] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:03.021930 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 778/1527] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:07:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:07:07.113838 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5242c-a22f-44b9-9848-76fc1b4356e7 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:07.114387 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5242c-a22f-44b9-9848-76fc1b4356e7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:07.114387 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5242c-a22f-44b9-9848-76fc1b4356e7 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:07.115675 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 750/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:07:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:07.121666 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a71c1-68e5-42e0-8f54-9796e5a76e64 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:07.122062 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a71c1-68e5-42e0-8f54-9796e5a76e64 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:07.122239 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a71c1-68e5-42e0-8f54-9796e5a76e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:07.141795 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-7b5a71c1-68e5-42e0-8f54-9796e5a76e64 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:07.142998 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-7b5a71c1-68e5-42e0-8f54-9796e5a76e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:07.150956 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 779/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:07:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:07:07.224413 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e5242c-a22f-44b9-9848-76fc1b4356e7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:07.226433 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-59e5242c-a22f-44b9-9848-76fc1b4356e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:07.303570 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e5242c-a22f-44b9-9848-76fc1b4356e7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:07.307073 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d21c57f-e1c8-4761-a3fe-10542c6aeee1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:07.307685 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21c57f-e1c8-4761-a3fe-10542c6aeee1 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:07.307969 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21c57f-e1c8-4761-a3fe-10542c6aeee1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:07.308242 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21c57f-e1c8-4761-a3fe-10542c6aeee1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:07.318904 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d21c57f-e1c8-4761-a3fe-10542c6aeee1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:07.320322 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d21c57f-e1c8-4761-a3fe-10542c6aeee1 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:07.330070 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 751/1530] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:07:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:07:09.334630 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758deb81-69ff-4ec0-817f-a20550983718 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:09.337344 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758deb81-69ff-4ec0-817f-a20550983718 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:09.337344 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758deb81-69ff-4ec0-817f-a20550983718 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:09.337344 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 780/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:07:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:09.342948 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21812be-d3b9-43bd-ad39-433b705cc03e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:09.343513 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21812be-d3b9-43bd-ad39-433b705cc03e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:09.343695 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21812be-d3b9-43bd-ad39-433b705cc03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:09.365149 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c21812be-d3b9-43bd-ad39-433b705cc03e None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:09.366347 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c21812be-d3b9-43bd-ad39-433b705cc03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:09.374268 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 752/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:07:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:07:09.411266 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758deb81-69ff-4ec0-817f-a20550983718 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:09.412988 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-758deb81-69ff-4ec0-817f-a20550983718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:09.425842 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758deb81-69ff-4ec0-817f-a20550983718 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:09.429362 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812704b5-7771-48c2-87a8-5417356e02e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:09.430087 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812704b5-7771-48c2-87a8-5417356e02e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:09.430380 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812704b5-7771-48c2-87a8-5417356e02e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:09.430659 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812704b5-7771-48c2-87a8-5417356e02e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:09.439645 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812704b5-7771-48c2-87a8-5417356e02e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:09.441115 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812704b5-7771-48c2-87a8-5417356e02e6 service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:09.451246 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 781/1533] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:07:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:07:13.038840 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b84e2-cded-41ab-9b17-b0d07106ccaa None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:13.039388 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b84e2-cded-41ab-9b17-b0d07106ccaa None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:13.039388 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b84e2-cded-41ab-9b17-b0d07106ccaa None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:13.040771 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 753/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:13.046529 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f88351-40f6-402b-ad5b-5d5d68e9a2d5 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:13.046955 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f88351-40f6-402b-ad5b-5d5d68e9a2d5 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:13.047107 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f88351-40f6-402b-ad5b-5d5d68e9a2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:13.066785 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-f8f88351-40f6-402b-ad5b-5d5d68e9a2d5 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:13.068049 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-f8f88351-40f6-402b-ad5b-5d5d68e9a2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:13.075850 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 782/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:07:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:07:13.381628 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b84e2-cded-41ab-9b17-b0d07106ccaa None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:13.383359 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-147b84e2-cded-41ab-9b17-b0d07106ccaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:13.447892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:13.461344 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b84e2-cded-41ab-9b17-b0d07106ccaa None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:13.464968 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46160f56-7c01-44f0-9a98-28962df5a728 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:13.466111 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46160f56-7c01-44f0-9a98-28962df5a728 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:13.466397 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46160f56-7c01-44f0-9a98-28962df5a728 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:13.466680 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46160f56-7c01-44f0-9a98-28962df5a728 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:13.475909 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46160f56-7c01-44f0-9a98-28962df5a728 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:13.477384 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46160f56-7c01-44f0-9a98-28962df5a728 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:13.487509 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 754/1536] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:07:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:07:15.607104 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0e1ce-6b58-45e8-84c1-b6065c448109 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:15.607742 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0e1ce-6b58-45e8-84c1-b6065c448109 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:15.607742 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0e1ce-6b58-45e8-84c1-b6065c448109 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:15.609070 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 783/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 14:07:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:07:15.614586 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3aab72-1d6c-44e5-b0ca-fa3bfb3f29d0 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:15.615013 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3aab72-1d6c-44e5-b0ca-fa3bfb3f29d0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:15.615121 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3aab72-1d6c-44e5-b0ca-fa3bfb3f29d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:15.634596 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-0a3aab72-1d6c-44e5-b0ca-fa3bfb3f29d0 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:07:15.635799 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0a3aab72-1d6c-44e5-b0ca-fa3bfb3f29d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:15.644174 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 755/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 14:07:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:07:15.947904 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0e1ce-6b58-45e8-84c1-b6065c448109 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:07:15.949611 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-30c0e1ce-6b58-45e8-84c1-b6065c448109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:07:15.962473 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0e1ce-6b58-45e8-84c1-b6065c448109 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:07:15.966366 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c43e9-2d83-48a3-98d7-a92b0a62f131 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:07:15.966968 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c43e9-2d83-48a3-98d7-a92b0a62f131 service cinder] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:07:15.967251 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c43e9-2d83-48a3-98d7-a92b0a62f131 service cinder] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:07:15.967529 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c43e9-2d83-48a3-98d7-a92b0a62f131 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:07:15.976596 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952c43e9-2d83-48a3-98d7-a92b0a62f131 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:07:15.978012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952c43e9-2d83-48a3-98d7-a92b0a62f131 service cinder] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:07:15.987743 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 784/1539] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:07:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:07:25.629948 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=11 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:07:25.983649 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=33 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:48.327522 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04659e7c-0f59-4d44-ac2f-69791b4f9e98 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:48.328696 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04659e7c-0f59-4d44-ac2f-69791b4f9e98 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:48.329231 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04659e7c-0f59-4d44-ac2f-69791b4f9e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:48.354203 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443466c-dc74-472b-9dc9-7515fffc6d06 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:48.355678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443466c-dc74-472b-9dc9-7515fffc6d06 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:48.356196 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443466c-dc74-472b-9dc9-7515fffc6d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:48.359102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-04659e7c-0f59-4d44-ac2f-69791b4f9e98 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:48.360426 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-04659e7c-0f59-4d44-ac2f-69791b4f9e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:48.375115 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 756/1540] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:08:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:48.388854 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-e443466c-dc74-472b-9dc9-7515fffc6d06 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:48.390577 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-e443466c-dc74-472b-9dc9-7515fffc6d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:48.399149 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9751b0-fdc4-4388-aca7-af984b2fad5e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:48.399778 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9751b0-fdc4-4388-aca7-af984b2fad5e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:48.400302 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9751b0-fdc4-4388-aca7-af984b2fad5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:48.404458 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 785/1541] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:08:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:48.419029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b8ee6-f2ae-4ba4-a47e-90b15a5292d2 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:48.419632 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b8ee6-f2ae-4ba4-a47e-90b15a5292d2 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:48.420123 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b8ee6-f2ae-4ba4-a47e-90b15a5292d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:48.435306 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-0b9751b0-fdc4-4388-aca7-af984b2fad5e None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:48.437007 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0b9751b0-fdc4-4388-aca7-af984b2fad5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:48.456783 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 757/1542] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:08:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:08:48.457002 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-087b8ee6-f2ae-4ba4-a47e-90b15a5292d2 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:48.458362 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-087b8ee6-f2ae-4ba4-a47e-90b15a5292d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:48.458665 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5382cf-9e27-498b-b1e9-676665bd2dcc None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:48.459115 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5382cf-9e27-498b-b1e9-676665bd2dcc None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:48.459651 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5382cf-9e27-498b-b1e9-676665bd2dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:48.478622 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 786/1543] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:08:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:08:48.479802 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b8ee6-f2ae-4ba4-a47e-90b15a5292d2 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:48.493511 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-aa5382cf-9e27-498b-b1e9-676665bd2dcc None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:48.495003 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-aa5382cf-9e27-498b-b1e9-676665bd2dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:48.498257 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b8ee6-f2ae-4ba4-a47e-90b15a5292d2 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:48.509047 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 758/1544] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:08:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:48.510276 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5382cf-9e27-498b-b1e9-676665bd2dcc None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:48.512006 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4e29ca-8816-4b18-8bce-4d387cde03c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:48.512935 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e29ca-8816-4b18-8bce-4d387cde03c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:48.513323 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e29ca-8816-4b18-8bce-4d387cde03c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:48.513707 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e29ca-8816-4b18-8bce-4d387cde03c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:48.514527 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4e29ca-8816-4b18-8bce-4d387cde03c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:08:48.514939 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4e29ca-8816-4b18-8bce-4d387cde03c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:48.516061 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: WARNING py.warnings [None req-db4e29ca-8816-4b18-8bce-4d387cde03c6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:08:48.516061 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: warnings.warn(msg) Aug 01 14:08:48.516061 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: Aug 01 14:08:48.524504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4e29ca-8816-4b18-8bce-4d387cde03c6 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:48.530184 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5382cf-9e27-498b-b1e9-676665bd2dcc None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:48.539091 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 787/1545] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:48.539863 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e1a70-31f9-4f9d-b8f0-42285b33206e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:48.540603 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc566d5-6bf1-4142-a997-bb957ad5fe1a None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:48.540889 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e1a70-31f9-4f9d-b8f0-42285b33206e admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:48.541081 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc566d5-6bf1-4142-a997-bb957ad5fe1a None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:48.541461 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc566d5-6bf1-4142-a997-bb957ad5fe1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:48.541675 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e1a70-31f9-4f9d-b8f0-42285b33206e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:48.542110 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e1a70-31f9-4f9d-b8f0-42285b33206e admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:48.543526 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00e1a70-31f9-4f9d-b8f0-42285b33206e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:08:48.544010 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00e1a70-31f9-4f9d-b8f0-42285b33206e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:48.546208 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00e1a70-31f9-4f9d-b8f0-42285b33206e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:48.570338 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 759/1546] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:48.571446 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e1a70-31f9-4f9d-b8f0-42285b33206e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:48.589692 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-bbc566d5-6bf1-4142-a997-bb957ad5fe1a None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:48.590162 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e1a70-31f9-4f9d-b8f0-42285b33206e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:48.591339 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-bbc566d5-6bf1-4142-a997-bb957ad5fe1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:48.598540 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537f5b29-68b1-40b8-9ca7-f3414306c981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:48.599723 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537f5b29-68b1-40b8-9ca7-f3414306c981 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:48.600259 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537f5b29-68b1-40b8-9ca7-f3414306c981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:48.600853 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537f5b29-68b1-40b8-9ca7-f3414306c981 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:48.601716 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537f5b29-68b1-40b8-9ca7-f3414306c981 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:48.603065 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: WARNING py.warnings [None req-537f5b29-68b1-40b8-9ca7-f3414306c981 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:08:48.603065 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: warnings.warn(msg) Aug 01 14:08:48.603065 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: Aug 01 14:08:48.604839 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537f5b29-68b1-40b8-9ca7-f3414306c981 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:48.611678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 788/1547] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:08:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:48.611678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc566d5-6bf1-4142-a997-bb957ad5fe1a None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:48.634242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc566d5-6bf1-4142-a997-bb957ad5fe1a None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:48.657783 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a43fd-4264-4800-973e-1e84bc4059f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:48.658440 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a43fd-4264-4800-973e-1e84bc4059f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:48.658728 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a43fd-4264-4800-973e-1e84bc4059f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:48.659014 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a43fd-4264-4800-973e-1e84bc4059f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:48.659692 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a43fd-4264-4800-973e-1e84bc4059f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:48.664146 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a43fd-4264-4800-973e-1e84bc4059f3 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:48.701477 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 760/1548] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:48 2022] POST /identity/v3/projects => generated 379 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:48.702371 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537f5b29-68b1-40b8-9ca7-f3414306c981 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:48.731749 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 789/1549] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:48 2022] POST /identity/v3/projects => generated 381 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:48.733098 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c9db0-198d-4b59-b295-8864ad9f3fbf None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:48.733392 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c9db0-198d-4b59-b295-8864ad9f3fbf None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:48.733926 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c9db0-198d-4b59-b295-8864ad9f3fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:48.759709 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-fd8c9db0-198d-4b59-b295-8864ad9f3fbf None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:48.762098 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-fd8c9db0-198d-4b59-b295-8864ad9f3fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:48.794972 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537f5b29-68b1-40b8-9ca7-f3414306c981 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:48.828691 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 790/1550] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:08:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:48.829562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c9db0-198d-4b59-b295-8864ad9f3fbf None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:48.838682 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c9db0-198d-4b59-b295-8864ad9f3fbf None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:48.844041 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411839cd-de76-463c-b441-6d80b5c55ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:48.844619 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411839cd-de76-463c-b441-6d80b5c55ac2 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:48.844903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411839cd-de76-463c-b441-6d80b5c55ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:48.845442 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411839cd-de76-463c-b441-6d80b5c55ac2 admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:48.846098 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411839cd-de76-463c-b441-6d80b5c55ac2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:48.847126 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: WARNING py.warnings [None req-411839cd-de76-463c-b441-6d80b5c55ac2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:08:48.847126 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: warnings.warn(msg) Aug 01 14:08:48.847126 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: Aug 01 14:08:48.848383 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411839cd-de76-463c-b441-6d80b5c55ac2 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:48.858948 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3582cbec-9b49-4025-ba0e-2b60bac3fc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:48.859288 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3582cbec-9b49-4025-ba0e-2b60bac3fc6f admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:48.859576 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3582cbec-9b49-4025-ba0e-2b60bac3fc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:48.859861 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3582cbec-9b49-4025-ba0e-2b60bac3fc6f admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:48.860637 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3582cbec-9b49-4025-ba0e-2b60bac3fc6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:08:48.860950 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3582cbec-9b49-4025-ba0e-2b60bac3fc6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:48.862756 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3582cbec-9b49-4025-ba0e-2b60bac3fc6f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:48.870339 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 791/1551] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:48 2022] POST /identity/v3/users => generated 290 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:48.871667 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f4721-5b36-4864-8a26-5f67e9b93036 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:48.871996 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f4721-5b36-4864-8a26-5f67e9b93036 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:48.872278 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f4721-5b36-4864-8a26-5f67e9b93036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:48.885675 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 761/1552] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:48.885675 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3582cbec-9b49-4025-ba0e-2b60bac3fc6f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:48.890900 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3582cbec-9b49-4025-ba0e-2b60bac3fc6f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:48.891926 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-df0f4721-5b36-4864-8a26-5f67e9b93036 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:48.893200 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-df0f4721-5b36-4864-8a26-5f67e9b93036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:48.895726 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d18d69-b35c-45b5-a4b0-d85533fbad3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:48.896344 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d18d69-b35c-45b5-a4b0-d85533fbad3b admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:48.896661 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d18d69-b35c-45b5-a4b0-d85533fbad3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:48.896944 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d18d69-b35c-45b5-a4b0-d85533fbad3b admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:48.897612 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d18d69-b35c-45b5-a4b0-d85533fbad3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:48.899522 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d18d69-b35c-45b5-a4b0-d85533fbad3b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:48.906522 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 792/1553] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:08:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:48.906522 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c0ade-65eb-48de-b999-56743d708136 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:48.906522 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c0ade-65eb-48de-b999-56743d708136 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:48.906522 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c0ade-65eb-48de-b999-56743d708136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:48.924500 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 762/1554] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:48 2022] POST /identity/v3/users => generated 291 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:48.925833 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9f500-19f3-42ff-af1e-609a8e090dc7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:48.926128 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9f500-19f3-42ff-af1e-609a8e090dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:48.926415 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9f500-19f3-42ff-af1e-609a8e090dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:48.928217 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-292c0ade-65eb-48de-b999-56743d708136 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:48.929459 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-292c0ade-65eb-48de-b999-56743d708136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:48.938087 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 793/1555] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:08:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:48.939405 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0746b5-ea1c-4b34-83c3-c52aa1e3782f None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:48.939697 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0746b5-ea1c-4b34-83c3-c52aa1e3782f None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:48.939980 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0746b5-ea1c-4b34-83c3-c52aa1e3782f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:48.951187 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8ab9f500-19f3-42ff-af1e-609a8e090dc7 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:48.952460 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8ab9f500-19f3-42ff-af1e-609a8e090dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:48.959759 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-ab0746b5-ea1c-4b34-83c3-c52aa1e3782f None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:48.960664 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 763/1556] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:08:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:48.960968 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-ab0746b5-ea1c-4b34-83c3-c52aa1e3782f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:48.961944 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babfa09-80d5-452f-be00-9f85ef69a1c7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:48.962234 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babfa09-80d5-452f-be00-9f85ef69a1c7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:48.962622 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babfa09-80d5-452f-be00-9f85ef69a1c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:48.975472 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 794/1557] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:08:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:48.983350 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0746b5-ea1c-4b34-83c3-c52aa1e3782f None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:48.984287 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4babfa09-80d5-452f-be00-9f85ef69a1c7 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:48.985686 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4babfa09-80d5-452f-be00-9f85ef69a1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:48.991317 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0746b5-ea1c-4b34-83c3-c52aa1e3782f None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:48.994162 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 764/1558] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:08:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:48.995354 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4babfa09-80d5-452f-be00-9f85ef69a1c7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:49.000926 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c8e18-a1cb-47b1-b8ec-6aa742a2bc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:49.001671 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c8e18-a1cb-47b1-b8ec-6aa742a2bc22 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:49.002069 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c8e18-a1cb-47b1-b8ec-6aa742a2bc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:49.002451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c8e18-a1cb-47b1-b8ec-6aa742a2bc22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:49.003254 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85c8e18-a1cb-47b1-b8ec-6aa742a2bc22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:49.005001 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85c8e18-a1cb-47b1-b8ec-6aa742a2bc22 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:49.008114 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4babfa09-80d5-452f-be00-9f85ef69a1c7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:49.013028 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff362261-7e5b-492b-904d-10e56c19efbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:49.013672 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff362261-7e5b-492b-904d-10e56c19efbb admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:49.013954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff362261-7e5b-492b-904d-10e56c19efbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:49.014250 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff362261-7e5b-492b-904d-10e56c19efbb admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:49.014902 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff362261-7e5b-492b-904d-10e56c19efbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:49.016882 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff362261-7e5b-492b-904d-10e56c19efbb admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:49.017121 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 795/1559] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:49.018549 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f9242-0ccb-40e2-b97a-1b9faa7e4f35 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:49.018843 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f9242-0ccb-40e2-b97a-1b9faa7e4f35 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:49.019122 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f9242-0ccb-40e2-b97a-1b9faa7e4f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:49.023259 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:49.041409 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-6c1f9242-0ccb-40e2-b97a-1b9faa7e4f35 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:49.043256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-6c1f9242-0ccb-40e2-b97a-1b9faa7e4f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:49.051894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 796/1560] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:08:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:49.054010 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 765/1561] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:48 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:49.055534 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eff7fa-d6ae-42bd-b97d-4791a74c7ec0 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:49.055827 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eff7fa-d6ae-42bd-b97d-4791a74c7ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:49.056104 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eff7fa-d6ae-42bd-b97d-4791a74c7ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:49.056365 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f9242-0ccb-40e2-b97a-1b9faa7e4f35 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:49.066807 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f9242-0ccb-40e2-b97a-1b9faa7e4f35 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:49.078204 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-25eff7fa-d6ae-42bd-b97d-4791a74c7ec0 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:49.079408 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-25eff7fa-d6ae-42bd-b97d-4791a74c7ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:49.115445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:49.127674 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8772285e-a9eb-4c7c-8123-9232df1aa062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:49.128301 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772285e-a9eb-4c7c-8123-9232df1aa062 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:49.128617 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772285e-a9eb-4c7c-8123-9232df1aa062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:49.128898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772285e-a9eb-4c7c-8123-9232df1aa062 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:49.129597 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8772285e-a9eb-4c7c-8123-9232df1aa062 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:49.131192 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8772285e-a9eb-4c7c-8123-9232df1aa062 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:49.139150 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 766/1562] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:08:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:49.142815 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871559ea-0266-4369-bbf3-ed97bf080e47 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:49.142815 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871559ea-0266-4369-bbf3-ed97bf080e47 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:49.142815 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871559ea-0266-4369-bbf3-ed97bf080e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:49.143077 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 797/1563] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:49.143077 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9249216-0158-46cc-abe8-a521e8fa9f27 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:49.143077 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9249216-0158-46cc-abe8-a521e8fa9f27 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:49.143077 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9249216-0158-46cc-abe8-a521e8fa9f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:49.161427 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-871559ea-0266-4369-bbf3-ed97bf080e47 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:49.162597 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-a9249216-0158-46cc-abe8-a521e8fa9f27 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:49.162736 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-871559ea-0266-4369-bbf3-ed97bf080e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:49.163749 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-a9249216-0158-46cc-abe8-a521e8fa9f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:49.171378 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 767/1564] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:08:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:49.175465 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 798/1565] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:08:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:49.175615 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871559ea-0266-4369-bbf3-ed97bf080e47 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:49.175715 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9249216-0158-46cc-abe8-a521e8fa9f27 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:49.185058 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9249216-0158-46cc-abe8-a521e8fa9f27 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:49.185293 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871559ea-0266-4369-bbf3-ed97bf080e47 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:49.189928 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf91834-a2bb-42f1-b868-fddbf46b39eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:49.190472 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b4d58-d394-4cc9-8a20-24e790aa977f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:49.190599 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf91834-a2bb-42f1-b868-fddbf46b39eb admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:49.190829 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf91834-a2bb-42f1-b868-fddbf46b39eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:49.191082 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b4d58-d394-4cc9-8a20-24e790aa977f admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:49.191180 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf91834-a2bb-42f1-b868-fddbf46b39eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:49.191362 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b4d58-d394-4cc9-8a20-24e790aa977f admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:49.191636 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b4d58-d394-4cc9-8a20-24e790aa977f admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:49.191835 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf91834-a2bb-42f1-b868-fddbf46b39eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:08:49.192146 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf91834-a2bb-42f1-b868-fddbf46b39eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:49.192330 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b4d58-d394-4cc9-8a20-24e790aa977f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:49.193801 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf91834-a2bb-42f1-b868-fddbf46b39eb admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:49.193940 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b4d58-d394-4cc9-8a20-24e790aa977f admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:49.204402 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 799/1566] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:49.205201 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b4d58-d394-4cc9-8a20-24e790aa977f admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:49.205614 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 768/1567] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:08:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:49.209367 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf91834-a2bb-42f1-b868-fddbf46b39eb admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:49.215562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b4d58-d394-4cc9-8a20-24e790aa977f admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:49.219737 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf91834-a2bb-42f1-b868-fddbf46b39eb admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:49.220556 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4b0708-1bb9-4986-8e53-7dd4ea3ae441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:49.221169 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b0708-1bb9-4986-8e53-7dd4ea3ae441 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:49.221486 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b0708-1bb9-4986-8e53-7dd4ea3ae441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:49.221824 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b0708-1bb9-4986-8e53-7dd4ea3ae441 admin admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:49.222460 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4b0708-1bb9-4986-8e53-7dd4ea3ae441 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:08:49.222765 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4b0708-1bb9-4986-8e53-7dd4ea3ae441 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:49.224351 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4b0708-1bb9-4986-8e53-7dd4ea3ae441 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:49.224786 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c27cfb-eade-4842-9bf9-03996be0fd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:49.225440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c27cfb-eade-4842-9bf9-03996be0fd3e admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:49.225829 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c27cfb-eade-4842-9bf9-03996be0fd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:49.226005 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c27cfb-eade-4842-9bf9-03996be0fd3e admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:49.226668 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c27cfb-eade-4842-9bf9-03996be0fd3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:49.228593 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c27cfb-eade-4842-9bf9-03996be0fd3e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:49.237715 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 800/1568] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:08:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:49.238562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4b0708-1bb9-4986-8e53-7dd4ea3ae441 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:49.248759 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4b0708-1bb9-4986-8e53-7dd4ea3ae441 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:49.252640 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 769/1569] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:49 2022] POST /identity/v3/users => generated 293 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:49.255662 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c27cfb-eade-4842-9bf9-03996be0fd3e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:49.255761 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d07fead-a669-46cf-b845-b0b8c109d72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:49.255761 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07fead-a669-46cf-b845-b0b8c109d72e admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:49.255761 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07fead-a669-46cf-b845-b0b8c109d72e admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:49.255761 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07fead-a669-46cf-b845-b0b8c109d72e admin admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:49.256074 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d07fead-a669-46cf-b845-b0b8c109d72e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:08:49.256386 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d07fead-a669-46cf-b845-b0b8c109d72e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:49.258439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d07fead-a669-46cf-b845-b0b8c109d72e admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:49.263184 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c27cfb-eade-4842-9bf9-03996be0fd3e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:49.268228 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bea6f5-c756-4965-988b-62168f479294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:49.268860 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bea6f5-c756-4965-988b-62168f479294 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:49.269141 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bea6f5-c756-4965-988b-62168f479294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:49.269416 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bea6f5-c756-4965-988b-62168f479294 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:49.270107 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bea6f5-c756-4965-988b-62168f479294 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:49.270505 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 801/1570] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:08:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:49.271374 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d07fead-a669-46cf-b845-b0b8c109d72e admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:49.271766 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bea6f5-c756-4965-988b-62168f479294 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:49.280855 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d07fead-a669-46cf-b845-b0b8c109d72e admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:49.285512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab46555-93c7-41dd-bfc8-1fd047da2806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:49.286131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab46555-93c7-41dd-bfc8-1fd047da2806 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:49.286409 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab46555-93c7-41dd-bfc8-1fd047da2806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:49.286681 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab46555-93c7-41dd-bfc8-1fd047da2806 admin admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:49.286917 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 770/1571] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:49.287364 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab46555-93c7-41dd-bfc8-1fd047da2806 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:08:49.287729 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab46555-93c7-41dd-bfc8-1fd047da2806 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:49.287854 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bea6f5-c756-4965-988b-62168f479294 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:49.289331 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab46555-93c7-41dd-bfc8-1fd047da2806 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:49.300174 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bea6f5-c756-4965-988b-62168f479294 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:49.305582 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 802/1572] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:08:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:49.305582 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab46555-93c7-41dd-bfc8-1fd047da2806 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:49.307286 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f1e7b-e84c-4539-8a54-b8248be57174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:49.307897 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f1e7b-e84c-4539-8a54-b8248be57174 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:49.308804 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f1e7b-e84c-4539-8a54-b8248be57174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:49.309091 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f1e7b-e84c-4539-8a54-b8248be57174 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:49.309786 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f1e7b-e84c-4539-8a54-b8248be57174 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:08:49.310093 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f1e7b-e84c-4539-8a54-b8248be57174 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:49.311666 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f1e7b-e84c-4539-8a54-b8248be57174 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:49.313890 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab46555-93c7-41dd-bfc8-1fd047da2806 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:49.318442 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849ba64-6c40-474c-9b8d-8b69f9b63511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:49.319087 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849ba64-6c40-474c-9b8d-8b69f9b63511 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:49.319373 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849ba64-6c40-474c-9b8d-8b69f9b63511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:49.319652 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849ba64-6c40-474c-9b8d-8b69f9b63511 admin admin] PATH_INFO: `/v3/projects/c68666a08e3142de947cfa44ec5816a9/users/822ea58eeb4e4d29b132964c0e3355db/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:49.323758 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 771/1573] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:08:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:08:49.324550 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f1e7b-e84c-4539-8a54-b8248be57174 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:49.333793 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f1e7b-e84c-4539-8a54-b8248be57174 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:49.338173 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f849ba64-6c40-474c-9b8d-8b69f9b63511 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c68666a08e3142de947cfa44ec5816a9, user_id=822ea58eeb4e4d29b132964c0e3355db, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:49.338449 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7361999-b86f-4a57-ad46-e07f7157aae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:49.339050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7361999-b86f-4a57-ad46-e07f7157aae0 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:49.339364 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7361999-b86f-4a57-ad46-e07f7157aae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:49.340109 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7361999-b86f-4a57-ad46-e07f7157aae0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:49.340889 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f849ba64-6c40-474c-9b8d-8b69f9b63511 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:49.341256 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7361999-b86f-4a57-ad46-e07f7157aae0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:49.343134 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7361999-b86f-4a57-ad46-e07f7157aae0 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:49.356244 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 803/1574] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:49 2022] PUT /identity/v3/projects/c68666a08e3142de947cfa44ec5816a9/users/822ea58eeb4e4d29b132964c0e3355db/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:49.361186 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849ba64-6c40-474c-9b8d-8b69f9b63511 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:49.378344 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 772/1575] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:49 2022] POST /identity/v3/projects => generated 367 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:49.378344 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7361999-b86f-4a57-ad46-e07f7157aae0 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:49.427524 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849ba64-6c40-474c-9b8d-8b69f9b63511 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:49.439634 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7361999-b86f-4a57-ad46-e07f7157aae0 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:49.444179 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97059a60-f9c1-47b5-a5d0-d7c514d846ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:49.444824 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97059a60-f9c1-47b5-a5d0-d7c514d846ff admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:49.445112 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97059a60-f9c1-47b5-a5d0-d7c514d846ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:49.445388 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97059a60-f9c1-47b5-a5d0-d7c514d846ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:49.446081 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97059a60-f9c1-47b5-a5d0-d7c514d846ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:49.447672 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97059a60-f9c1-47b5-a5d0-d7c514d846ff admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:49.449964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:49.460661 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 773/1576] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:49.461487 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97059a60-f9c1-47b5-a5d0-d7c514d846ff admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:49.471779 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97059a60-f9c1-47b5-a5d0-d7c514d846ff admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:49.476412 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39404231-a799-4f04-b584-8e4df34f50da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:49.477060 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39404231-a799-4f04-b584-8e4df34f50da admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:49.477347 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39404231-a799-4f04-b584-8e4df34f50da admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:49.477624 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39404231-a799-4f04-b584-8e4df34f50da admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:49.478288 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39404231-a799-4f04-b584-8e4df34f50da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:49.480977 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39404231-a799-4f04-b584-8e4df34f50da admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:49.485676 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822e3cc4-4514-4491-84c4-dba2639cacc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:49.486305 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e3cc4-4514-4491-84c4-dba2639cacc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:49.486592 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e3cc4-4514-4491-84c4-dba2639cacc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:49.486876 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e3cc4-4514-4491-84c4-dba2639cacc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:49.487540 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822e3cc4-4514-4491-84c4-dba2639cacc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:49.489486 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822e3cc4-4514-4491-84c4-dba2639cacc7 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:49.508672 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 774/1577] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:49 2022] POST /identity/v3/projects => generated 389 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:49.509860 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39404231-a799-4f04-b584-8e4df34f50da admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:49.513573 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:49.520115 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 804/1578] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:49 2022] POST /identity/v3/projects => generated 373 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:49.520115 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822e3cc4-4514-4491-84c4-dba2639cacc7 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:49.575406 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39404231-a799-4f04-b584-8e4df34f50da admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:49.580114 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc6efe-2b2b-48c2-b247-4e3c31926486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:49.580814 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc6efe-2b2b-48c2-b247-4e3c31926486 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:49.581101 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc6efe-2b2b-48c2-b247-4e3c31926486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:49.581382 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc6efe-2b2b-48c2-b247-4e3c31926486 admin admin] PATH_INFO: `/v3/projects/0993d5226b264f3fa129a55e74f0eb5a/users/73f955986c3b4d77a6f5cd1a823155a8/roles/926b9f36c24847c582c84af904250abc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:49.583954 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822e3cc4-4514-4491-84c4-dba2639cacc7 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:49.588782 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152fa31-2b22-4a90-9050-811575543e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:49.589430 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152fa31-2b22-4a90-9050-811575543e36 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:49.589717 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152fa31-2b22-4a90-9050-811575543e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:49.589995 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152fa31-2b22-4a90-9050-811575543e36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:49.590656 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9152fa31-2b22-4a90-9050-811575543e36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:49.592568 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9152fa31-2b22-4a90-9050-811575543e36 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:49.597118 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc6efe-2b2b-48c2-b247-4e3c31926486 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0993d5226b264f3fa129a55e74f0eb5a, user_id=73f955986c3b4d77a6f5cd1a823155a8, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:49.598423 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: WARNING py.warnings [None req-a1cc6efe-2b2b-48c2-b247-4e3c31926486 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:08:49.598423 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: warnings.warn(msg) Aug 01 14:08:49.598423 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: Aug 01 14:08:49.600086 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc6efe-2b2b-48c2-b247-4e3c31926486 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:49.614402 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 775/1579] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:49 2022] PUT /identity/v3/projects/0993d5226b264f3fa129a55e74f0eb5a/users/73f955986c3b4d77a6f5cd1a823155a8/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:49.615195 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc6efe-2b2b-48c2-b247-4e3c31926486 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:49.625314 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 805/1580] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:49 2022] POST /identity/v3/projects => generated 391 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:49.626740 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182bdb06-296b-4b4a-928a-9388826dcd61 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:49.627030 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182bdb06-296b-4b4a-928a-9388826dcd61 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:49.627304 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182bdb06-296b-4b4a-928a-9388826dcd61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:49.646415 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-182bdb06-296b-4b4a-928a-9388826dcd61 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:49.647552 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-182bdb06-296b-4b4a-928a-9388826dcd61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:49.688500 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc6efe-2b2b-48c2-b247-4e3c31926486 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:49.706575 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 806/1581] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:08:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:49.707390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182bdb06-296b-4b4a-928a-9388826dcd61 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:49.717265 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182bdb06-296b-4b4a-928a-9388826dcd61 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:49.722196 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e2a28-0354-475d-82bc-7893c7641619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:49.722811 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e2a28-0354-475d-82bc-7893c7641619 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:49.723095 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e2a28-0354-475d-82bc-7893c7641619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:49.723375 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e2a28-0354-475d-82bc-7893c7641619 admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:49.724034 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e2a28-0354-475d-82bc-7893c7641619 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:49.725972 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e2a28-0354-475d-82bc-7893c7641619 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:49.749251 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90389968-624a-4ccf-810d-594e8f5af500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:49.749863 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90389968-624a-4ccf-810d-594e8f5af500 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:49.750147 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90389968-624a-4ccf-810d-594e8f5af500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:49.750424 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90389968-624a-4ccf-810d-594e8f5af500 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:49.750601 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 807/1582] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:49 2022] POST /identity/v3/users => generated 284 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:49.751512 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90389968-624a-4ccf-810d-594e8f5af500 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:49.751641 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e2a28-0354-475d-82bc-7893c7641619 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:49.753486 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90389968-624a-4ccf-810d-594e8f5af500 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:49.761944 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e2a28-0354-475d-82bc-7893c7641619 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:49.766141 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:49.766566 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ac627-10a1-420a-86be-9887fc944726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:49.767185 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ac627-10a1-420a-86be-9887fc944726 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:49.767473 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ac627-10a1-420a-86be-9887fc944726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:49.767763 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ac627-10a1-420a-86be-9887fc944726 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:49.768534 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720ac627-10a1-420a-86be-9887fc944726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:49.770306 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720ac627-10a1-420a-86be-9887fc944726 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:49.784517 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 808/1583] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:49.784517 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ac627-10a1-420a-86be-9887fc944726 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:49.787840 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 776/1584] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:49 2022] POST /identity/v3/projects => generated 377 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:49.788688 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90389968-624a-4ccf-810d-594e8f5af500 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:49.794716 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ac627-10a1-420a-86be-9887fc944726 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:49.847553 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:49.854475 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90389968-624a-4ccf-810d-594e8f5af500 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:49.855308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7069c275-b76f-45cf-8ee8-69270737cbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:49.856071 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069c275-b76f-45cf-8ee8-69270737cbe3 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:49.856354 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069c275-b76f-45cf-8ee8-69270737cbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:49.856661 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069c275-b76f-45cf-8ee8-69270737cbe3 admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:49.857324 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7069c275-b76f-45cf-8ee8-69270737cbe3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:49.859232 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7069c275-b76f-45cf-8ee8-69270737cbe3 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:49.859475 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60343ed7-eb7d-4a4b-88ef-7e47b3822148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:49.860100 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60343ed7-eb7d-4a4b-88ef-7e47b3822148 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:49.860388 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60343ed7-eb7d-4a4b-88ef-7e47b3822148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:49.860714 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60343ed7-eb7d-4a4b-88ef-7e47b3822148 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:49.861374 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60343ed7-eb7d-4a4b-88ef-7e47b3822148 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:49.863355 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60343ed7-eb7d-4a4b-88ef-7e47b3822148 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:49.881803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 809/1585] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:49 2022] POST /identity/v3/users => generated 295 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:49.883121 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3b35d-af39-4c9e-ac34-e9a06ba6790f None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:49.883377 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3b35d-af39-4c9e-ac34-e9a06ba6790f None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:49.883665 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3b35d-af39-4c9e-ac34-e9a06ba6790f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:49.891946 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 777/1586] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:49 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:49.892706 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60343ed7-eb7d-4a4b-88ef-7e47b3822148 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:49.903192 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60343ed7-eb7d-4a4b-88ef-7e47b3822148 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:49.908773 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88d5f4-3883-4666-9ac9-8b4a12036935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:49.909386 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88d5f4-3883-4666-9ac9-8b4a12036935 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:49.909674 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88d5f4-3883-4666-9ac9-8b4a12036935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:49.909956 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88d5f4-3883-4666-9ac9-8b4a12036935 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:49.910619 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd88d5f4-3883-4666-9ac9-8b4a12036935 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:49.912574 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd88d5f4-3883-4666-9ac9-8b4a12036935 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:49.913242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-bfd3b35d-af39-4c9e-ac34-e9a06ba6790f None None] MFA Rules not processed for user `73f955986c3b4d77a6f5cd1a823155a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:49.914456 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-bfd3b35d-af39-4c9e-ac34-e9a06ba6790f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:49.938524 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 778/1587] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:49 2022] POST /identity/v3/users => generated 296 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:49.938524 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88d5f4-3883-4666-9ac9-8b4a12036935 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:49.985560 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 810/1588] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:08:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:49.985560 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd3b35d-af39-4c9e-ac34-e9a06ba6790f None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:49.993097 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd3b35d-af39-4c9e-ac34-e9a06ba6790f None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:49.996354 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88d5f4-3883-4666-9ac9-8b4a12036935 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:49.997617 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb935e-66d0-46aa-bf26-62a593afccb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:49.998214 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb935e-66d0-46aa-bf26-62a593afccb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:49.998494 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb935e-66d0-46aa-bf26-62a593afccb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:49.998769 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb935e-66d0-46aa-bf26-62a593afccb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:49.999520 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedb935e-66d0-46aa-bf26-62a593afccb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:50.000101 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc568c-668b-4a57-aa6d-4b853c2342f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:50.000747 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc568c-668b-4a57-aa6d-4b853c2342f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:50.001037 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc568c-668b-4a57-aa6d-4b853c2342f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:50.001135 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedb935e-66d0-46aa-bf26-62a593afccb5 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:50.001348 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc568c-668b-4a57-aa6d-4b853c2342f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:50.010457 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabc568c-668b-4a57-aa6d-4b853c2342f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:50.010793 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 811/1589] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:50.012147 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb935e-66d0-46aa-bf26-62a593afccb5 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:50.012444 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabc568c-668b-4a57-aa6d-4b853c2342f3 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:50.022109 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 779/1590] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:50.022905 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc568c-668b-4a57-aa6d-4b853c2342f3 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:50.023105 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb935e-66d0-46aa-bf26-62a593afccb5 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:50.028789 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb2b4b-cf53-4f69-bfea-d60d8b749c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:50.029448 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb2b4b-cf53-4f69-bfea-d60d8b749c46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:50.029733 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb2b4b-cf53-4f69-bfea-d60d8b749c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:50.030014 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb2b4b-cf53-4f69-bfea-d60d8b749c46 admin admin] PATH_INFO: `/v3/projects/a3203f080a05422e83090a1f0a31d1a9/users/ca3cd1c2aec94b43aff0034b812a069d/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:50.033240 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc568c-668b-4a57-aa6d-4b853c2342f3 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:50.039429 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab904a8-1de9-43ff-bec4-079f3c7954c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:50.039788 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab904a8-1de9-43ff-bec4-079f3c7954c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:50.039968 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab904a8-1de9-43ff-bec4-079f3c7954c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:50.040248 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab904a8-1de9-43ff-bec4-079f3c7954c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:50.041268 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab904a8-1de9-43ff-bec4-079f3c7954c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:50.042937 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab904a8-1de9-43ff-bec4-079f3c7954c4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:50.046265 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fb2b4b-cf53-4f69-bfea-d60d8b749c46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3203f080a05422e83090a1f0a31d1a9, user_id=ca3cd1c2aec94b43aff0034b812a069d, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:50.049300 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fb2b4b-cf53-4f69-bfea-d60d8b749c46 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:50.065233 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 812/1591] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:50 2022] PUT /identity/v3/projects/a3203f080a05422e83090a1f0a31d1a9/users/ca3cd1c2aec94b43aff0034b812a069d/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:50.066688 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 780/1592] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:50 2022] POST /identity/v3/users => generated 289 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:50.067742 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab904a8-1de9-43ff-bec4-079f3c7954c4 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:50.069306 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb2b4b-cf53-4f69-bfea-d60d8b749c46 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:50.079955 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab904a8-1de9-43ff-bec4-079f3c7954c4 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:50.132931 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:50.145205 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb2b4b-cf53-4f69-bfea-d60d8b749c46 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:50.148511 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18500497-db6c-413c-9829-d395b4dff281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:50.149227 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18500497-db6c-413c-9829-d395b4dff281 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:50.149506 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18500497-db6c-413c-9829-d395b4dff281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:50.149781 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18500497-db6c-413c-9829-d395b4dff281 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:50.150500 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18500497-db6c-413c-9829-d395b4dff281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:50.150631 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1bcdd-12f1-4d07-9c1c-875900cca02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:50.151361 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1bcdd-12f1-4d07-9c1c-875900cca02a admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:50.151690 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1bcdd-12f1-4d07-9c1c-875900cca02a admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:50.152073 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1bcdd-12f1-4d07-9c1c-875900cca02a admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:50.152321 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18500497-db6c-413c-9829-d395b4dff281 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:50.153070 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f1bcdd-12f1-4d07-9c1c-875900cca02a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:50.155026 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f1bcdd-12f1-4d07-9c1c-875900cca02a admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:50.166367 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 781/1593] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:50.166367 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18500497-db6c-413c-9829-d395b4dff281 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:50.169701 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 813/1594] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:50.170495 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1bcdd-12f1-4d07-9c1c-875900cca02a admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:50.177334 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18500497-db6c-413c-9829-d395b4dff281 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:50.181320 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1bcdd-12f1-4d07-9c1c-875900cca02a admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:50.183710 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5def99a-277d-4c0c-88b0-ab859df33152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:50.184352 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5def99a-277d-4c0c-88b0-ab859df33152 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:50.184664 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5def99a-277d-4c0c-88b0-ab859df33152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:50.184954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5def99a-277d-4c0c-88b0-ab859df33152 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:50.185636 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5def99a-277d-4c0c-88b0-ab859df33152 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:50.187226 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5def99a-277d-4c0c-88b0-ab859df33152 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:50.187884 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0558ef-2842-43ef-8610-288b46a63fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:50.188672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0558ef-2842-43ef-8610-288b46a63fe3 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:50.189042 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0558ef-2842-43ef-8610-288b46a63fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:50.189646 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0558ef-2842-43ef-8610-288b46a63fe3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:50.190151 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0558ef-2842-43ef-8610-288b46a63fe3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:50.192105 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0558ef-2842-43ef-8610-288b46a63fe3 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:50.199981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 782/1595] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:50.200984 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5def99a-277d-4c0c-88b0-ab859df33152 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:50.217679 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5def99a-277d-4c0c-88b0-ab859df33152 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:50.224689 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fabeb96-6826-429e-b377-28099da3eb05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:50.226664 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fabeb96-6826-429e-b377-28099da3eb05 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:50.227386 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fabeb96-6826-429e-b377-28099da3eb05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:50.228285 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fabeb96-6826-429e-b377-28099da3eb05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:50.231881 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fabeb96-6826-429e-b377-28099da3eb05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:50.235969 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 814/1596] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:50 2022] POST /identity/v3/projects => generated 381 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:50.235969 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652252dc-9d08-4e28-9e18-6bdd28259c60 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:50.235969 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652252dc-9d08-4e28-9e18-6bdd28259c60 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:50.235969 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652252dc-9d08-4e28-9e18-6bdd28259c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:50.237642 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fabeb96-6826-429e-b377-28099da3eb05 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:50.248929 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 783/1597] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:50.250245 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fabeb96-6826-429e-b377-28099da3eb05 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:50.256840 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-652252dc-9d08-4e28-9e18-6bdd28259c60 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:50.258279 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-652252dc-9d08-4e28-9e18-6bdd28259c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:50.261528 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fabeb96-6826-429e-b377-28099da3eb05 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:50.269361 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:50.330200 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c94e84-462c-4be5-a96d-f112b757d468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:50.330869 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c94e84-462c-4be5-a96d-f112b757d468 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:50.331158 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c94e84-462c-4be5-a96d-f112b757d468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:50.331447 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c94e84-462c-4be5-a96d-f112b757d468 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:50.332218 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c94e84-462c-4be5-a96d-f112b757d468 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:50.333426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 815/1598] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:08:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:50.333883 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c94e84-462c-4be5-a96d-f112b757d468 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:50.334341 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652252dc-9d08-4e28-9e18-6bdd28259c60 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:50.347270 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652252dc-9d08-4e28-9e18-6bdd28259c60 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:50.349686 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 784/1599] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:50.350420 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c94e84-462c-4be5-a96d-f112b757d468 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:50.355492 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9808e2-352b-4da8-81f7-725d80bbf9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:50.356148 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9808e2-352b-4da8-81f7-725d80bbf9ba admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:50.356434 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9808e2-352b-4da8-81f7-725d80bbf9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:50.357108 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9808e2-352b-4da8-81f7-725d80bbf9ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:50.357718 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9808e2-352b-4da8-81f7-725d80bbf9ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:50.359368 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9808e2-352b-4da8-81f7-725d80bbf9ba admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:50.363587 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c94e84-462c-4be5-a96d-f112b757d468 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:50.371270 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 816/1600] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:50.371927 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9808e2-352b-4da8-81f7-725d80bbf9ba admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:50.384509 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9808e2-352b-4da8-81f7-725d80bbf9ba admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:50.385369 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae9159-7017-4bce-8666-f216875b3411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:50.386074 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae9159-7017-4bce-8666-f216875b3411 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:50.386507 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae9159-7017-4bce-8666-f216875b3411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:50.386855 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae9159-7017-4bce-8666-f216875b3411 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:50.387638 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbae9159-7017-4bce-8666-f216875b3411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:50.389456 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbae9159-7017-4bce-8666-f216875b3411 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:50.394443 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e2e148-f3ad-4fa8-a3ef-7e6a62c9cf9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:50.395344 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e2e148-f3ad-4fa8-a3ef-7e6a62c9cf9c admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:50.395789 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e2e148-f3ad-4fa8-a3ef-7e6a62c9cf9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:50.396229 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e2e148-f3ad-4fa8-a3ef-7e6a62c9cf9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:50.397310 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e2e148-f3ad-4fa8-a3ef-7e6a62c9cf9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:50.399115 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e2e148-f3ad-4fa8-a3ef-7e6a62c9cf9c admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:50.408771 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 785/1601] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:50.410207 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae9159-7017-4bce-8666-f216875b3411 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:50.413949 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 817/1602] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:50.415308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e2e148-f3ad-4fa8-a3ef-7e6a62c9cf9c admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:50.427085 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae9159-7017-4bce-8666-f216875b3411 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:50.431415 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e2e148-f3ad-4fa8-a3ef-7e6a62c9cf9c admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:50.435244 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93194603-18b9-484f-ab9a-d6010ca13d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:50.436527 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93194603-18b9-484f-ab9a-d6010ca13d82 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:50.436768 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93194603-18b9-484f-ab9a-d6010ca13d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:50.437288 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93194603-18b9-484f-ab9a-d6010ca13d82 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:50.438437 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93194603-18b9-484f-ab9a-d6010ca13d82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:50.441561 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93194603-18b9-484f-ab9a-d6010ca13d82 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:50.443261 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6acba40-4641-47dd-8b77-10914466cdf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:50.443962 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6acba40-4641-47dd-8b77-10914466cdf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:50.444247 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6acba40-4641-47dd-8b77-10914466cdf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:50.444875 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6acba40-4641-47dd-8b77-10914466cdf8 admin admin] PATH_INFO: `/v3/projects/97ba78c7fc184b549caa84cf6a4f0df1/users/14ef80d493364a4387b6556a339cbbb8/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:50.468170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6acba40-4641-47dd-8b77-10914466cdf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97ba78c7fc184b549caa84cf6a4f0df1, user_id=14ef80d493364a4387b6556a339cbbb8, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:50.472540 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 786/1603] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:50 2022] POST /identity/v3/users => generated 297 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:50.472699 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6acba40-4641-47dd-8b77-10914466cdf8 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:50.472825 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93194603-18b9-484f-ab9a-d6010ca13d82 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:50.488814 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 818/1604] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:50 2022] PUT /identity/v3/projects/97ba78c7fc184b549caa84cf6a4f0df1/users/14ef80d493364a4387b6556a339cbbb8/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:50.489680 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6acba40-4641-47dd-8b77-10914466cdf8 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:50.554316 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93194603-18b9-484f-ab9a-d6010ca13d82 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:50.566247 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6acba40-4641-47dd-8b77-10914466cdf8 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:50.572111 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a942b29e-6f24-4b28-8845-b6167fba9695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:50.572876 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942b29e-6f24-4b28-8845-b6167fba9695 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:50.573246 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942b29e-6f24-4b28-8845-b6167fba9695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:50.574175 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942b29e-6f24-4b28-8845-b6167fba9695 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:50.574988 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a942b29e-6f24-4b28-8845-b6167fba9695 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:50.576790 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a942b29e-6f24-4b28-8845-b6167fba9695 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:50.587545 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 819/1605] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:50.588167 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a942b29e-6f24-4b28-8845-b6167fba9695 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:50.598829 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a942b29e-6f24-4b28-8845-b6167fba9695 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:50.603932 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa725eb7-cdbf-4cbe-a9f1-2574003ea33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:50.604636 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa725eb7-cdbf-4cbe-a9f1-2574003ea33e admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:50.604953 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa725eb7-cdbf-4cbe-a9f1-2574003ea33e admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:50.605235 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa725eb7-cdbf-4cbe-a9f1-2574003ea33e admin admin] PATH_INFO: `/v3/projects/20a0f7842ef749f38d4bdf536f75d993/users/9c968d77125448a7860a47707224e1a4/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:50.610777 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:50.618285 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f013ca-fcbe-4c13-9fc8-39bc25c7a7c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:50.619589 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f013ca-fcbe-4c13-9fc8-39bc25c7a7c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:50.620367 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f013ca-fcbe-4c13-9fc8-39bc25c7a7c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:50.621161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f013ca-fcbe-4c13-9fc8-39bc25c7a7c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:50.621983 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa725eb7-cdbf-4cbe-a9f1-2574003ea33e admin admin] RBAC: Authorizing `identity:create_grant(project_id=20a0f7842ef749f38d4bdf536f75d993, user_id=9c968d77125448a7860a47707224e1a4, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:50.624876 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa725eb7-cdbf-4cbe-a9f1-2574003ea33e admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:50.632686 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f013ca-fcbe-4c13-9fc8-39bc25c7a7c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:50.634329 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f013ca-fcbe-4c13-9fc8-39bc25c7a7c1 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:50.641626 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 820/1606] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:50 2022] PUT /identity/v3/projects/20a0f7842ef749f38d4bdf536f75d993/users/9c968d77125448a7860a47707224e1a4/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:50.642608 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa725eb7-cdbf-4cbe-a9f1-2574003ea33e admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:50.654954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 787/1607] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:50.655876 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f013ca-fcbe-4c13-9fc8-39bc25c7a7c1 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:50.666624 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f013ca-fcbe-4c13-9fc8-39bc25c7a7c1 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:50.700527 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:50.724954 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa725eb7-cdbf-4cbe-a9f1-2574003ea33e admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:50.731518 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb04dcd8-358a-4804-b245-e8cba27986d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:50.733195 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04dcd8-358a-4804-b245-e8cba27986d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:50.733863 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04dcd8-358a-4804-b245-e8cba27986d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:50.734643 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04dcd8-358a-4804-b245-e8cba27986d2 admin admin] PATH_INFO: `/v3/projects/6d4402cabc4b41e0aec0ccc40cdd17cc/users/62cc969defd64837a20e9a7dbda5b4fa/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:50.739482 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d73de7-f64c-4ffc-a900-a36610ca287b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:50.739482 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d73de7-f64c-4ffc-a900-a36610ca287b admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:50.739482 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d73de7-f64c-4ffc-a900-a36610ca287b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:50.739482 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d73de7-f64c-4ffc-a900-a36610ca287b admin admin] PATH_INFO: `/v3/projects/670dbffe4aed43c6afc3899ff59b8040/users/f379b005418a4e399c3b72cbbef6e00d/roles/926b9f36c24847c582c84af904250abc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:50.763939 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb04dcd8-358a-4804-b245-e8cba27986d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d4402cabc4b41e0aec0ccc40cdd17cc, user_id=62cc969defd64837a20e9a7dbda5b4fa, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:50.765085 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d73de7-f64c-4ffc-a900-a36610ca287b admin admin] RBAC: Authorizing `identity:create_grant(project_id=670dbffe4aed43c6afc3899ff59b8040, user_id=f379b005418a4e399c3b72cbbef6e00d, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:50.766950 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb04dcd8-358a-4804-b245-e8cba27986d2 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:50.767988 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d73de7-f64c-4ffc-a900-a36610ca287b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:50.786383 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 788/1609] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:50 2022] PUT /identity/v3/projects/670dbffe4aed43c6afc3899ff59b8040/users/f379b005418a4e399c3b72cbbef6e00d/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:50.786610 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 821/1609] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:50 2022] PUT /identity/v3/projects/6d4402cabc4b41e0aec0ccc40cdd17cc/users/62cc969defd64837a20e9a7dbda5b4fa/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:50.787121 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d73de7-f64c-4ffc-a900-a36610ca287b admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:50.787797 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b97e32-d71f-4243-a3dd-3862b9803e78 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:50.788185 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b97e32-d71f-4243-a3dd-3862b9803e78 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:50.788569 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b97e32-d71f-4243-a3dd-3862b9803e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:50.838077 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-29b97e32-d71f-4243-a3dd-3862b9803e78 None None] MFA Rules not processed for user `14ef80d493364a4387b6556a339cbbb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:50.840260 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-29b97e32-d71f-4243-a3dd-3862b9803e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:50.885387 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d73de7-f64c-4ffc-a900-a36610ca287b admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:50.897405 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be0bf7-8944-42af-920c-1de9d7ccfed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:50.898055 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be0bf7-8944-42af-920c-1de9d7ccfed9 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:50.898339 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be0bf7-8944-42af-920c-1de9d7ccfed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:50.898664 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be0bf7-8944-42af-920c-1de9d7ccfed9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:50.899883 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23be0bf7-8944-42af-920c-1de9d7ccfed9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:50.902435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23be0bf7-8944-42af-920c-1de9d7ccfed9 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:50.918901 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 789/1610] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:50.919772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be0bf7-8944-42af-920c-1de9d7ccfed9 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:50.931836 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be0bf7-8944-42af-920c-1de9d7ccfed9 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:50.940500 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f79d78-8e3a-4a55-a9a0-46719d376cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:50.941329 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f79d78-8e3a-4a55-a9a0-46719d376cfb admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:50.941763 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f79d78-8e3a-4a55-a9a0-46719d376cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:50.942098 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f79d78-8e3a-4a55-a9a0-46719d376cfb admin admin] PATH_INFO: `/v3/projects/97ea1f5a5763441e985e38adfc4c81fb/users/047580fe85ef493d9de6c8831cb2d605/roles/926b9f36c24847c582c84af904250abc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:50.963866 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f79d78-8e3a-4a55-a9a0-46719d376cfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=97ea1f5a5763441e985e38adfc4c81fb, user_id=047580fe85ef493d9de6c8831cb2d605, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:50.966691 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f79d78-8e3a-4a55-a9a0-46719d376cfb admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:50.974919 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 822/1611] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:08:50 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:50.976298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72072b49-ee6a-4633-8490-dbf9758f434a None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:50.976743 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72072b49-ee6a-4633-8490-dbf9758f434a None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:50.977023 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72072b49-ee6a-4633-8490-dbf9758f434a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:51.014347 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 790/1612] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:50 2022] PUT /identity/v3/projects/97ea1f5a5763441e985e38adfc4c81fb/users/047580fe85ef493d9de6c8831cb2d605/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:51.018709 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc39dfd-79e6-4f1c-af5b-9c2fd9dcc79e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:51.019271 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-72072b49-ee6a-4633-8490-dbf9758f434a None None] MFA Rules not processed for user `9c968d77125448a7860a47707224e1a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:51.020276 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc39dfd-79e6-4f1c-af5b-9c2fd9dcc79e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:51.020616 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-72072b49-ee6a-4633-8490-dbf9758f434a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:51.021877 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc39dfd-79e6-4f1c-af5b-9c2fd9dcc79e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:51.068952 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4cc39dfd-79e6-4f1c-af5b-9c2fd9dcc79e None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:51.071465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4cc39dfd-79e6-4f1c-af5b-9c2fd9dcc79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:51.116190 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 823/1613] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:08:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:51.117541 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd836e66-4fda-4c71-9a47-e086abc88d99 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:51.118316 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd836e66-4fda-4c71-9a47-e086abc88d99 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:51.118657 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd836e66-4fda-4c71-9a47-e086abc88d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:51.174602 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:51.188461 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-bd836e66-4fda-4c71-9a47-e086abc88d99 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:51.191953 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-bd836e66-4fda-4c71-9a47-e086abc88d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:51.229387 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 791/1614] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:08:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:51.232982 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc39dfd-79e6-4f1c-af5b-9c2fd9dcc79e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:51.246709 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc39dfd-79e6-4f1c-af5b-9c2fd9dcc79e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:51.252995 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36be51-4d22-4bd0-9278-49c675ec2c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:51.254470 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36be51-4d22-4bd0-9278-49c675ec2c59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:51.254868 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36be51-4d22-4bd0-9278-49c675ec2c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:51.255363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36be51-4d22-4bd0-9278-49c675ec2c59 admin admin] PATH_INFO: `/v3/projects/6451650d419347f8b215b80ba3cb4bd6/users/47f31c68f0be45d3b415c3afcb2f3eb0/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:51.275005 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c36be51-4d22-4bd0-9278-49c675ec2c59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6451650d419347f8b215b80ba3cb4bd6, user_id=47f31c68f0be45d3b415c3afcb2f3eb0, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:51.275915 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:51.279602 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c36be51-4d22-4bd0-9278-49c675ec2c59 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:51.309684 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 792/1615] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:51 2022] PUT /identity/v3/projects/6451650d419347f8b215b80ba3cb4bd6/users/47f31c68f0be45d3b415c3afcb2f3eb0/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:51.310263 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36be51-4d22-4bd0-9278-49c675ec2c59 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:51.329325 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 824/1616] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:08:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:51.331146 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5451711-14fc-45c6-9892-f01b11a39638 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:51.331611 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5451711-14fc-45c6-9892-f01b11a39638 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:51.332037 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5451711-14fc-45c6-9892-f01b11a39638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:51.393841 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36be51-4d22-4bd0-9278-49c675ec2c59 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:51.398385 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682fb59-5925-4ba2-ab01-dafeb5f40da9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:51.399287 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682fb59-5925-4ba2-ab01-dafeb5f40da9 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:51.399447 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-b5451711-14fc-45c6-9892-f01b11a39638 None None] MFA Rules not processed for user `047580fe85ef493d9de6c8831cb2d605`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:51.399882 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682fb59-5925-4ba2-ab01-dafeb5f40da9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:51.400192 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682fb59-5925-4ba2-ab01-dafeb5f40da9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:51.405277 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-b5451711-14fc-45c6-9892-f01b11a39638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:51.475026 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682fb59-5925-4ba2-ab01-dafeb5f40da9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:51.476703 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682fb59-5925-4ba2-ab01-dafeb5f40da9 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:51.489939 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 793/1617] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:51 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:51.491424 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e6dff-56ce-4adc-add7-137482f466be None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:51.491795 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e6dff-56ce-4adc-add7-137482f466be None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:51.492157 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e6dff-56ce-4adc-add7-137482f466be None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:51.493950 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 794/1618] 10.222.0.48 () {54 vars in 962 bytes} [Mon Aug 1 14:08:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:08:51.494862 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e6dff-56ce-4adc-add7-137482f466be None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:51.574816 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 825/1619] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:08:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:51.575770 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5451711-14fc-45c6-9892-f01b11a39638 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:51.603481 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e6dff-56ce-4adc-add7-137482f466be None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:51.615785 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4bb6ea-6768-41ee-a2a8-a5f0731f06b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:51.626676 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4bb6ea-6768-41ee-a2a8-a5f0731f06b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:51.626676 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4bb6ea-6768-41ee-a2a8-a5f0731f06b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:51.629237 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4bb6ea-6768-41ee-a2a8-a5f0731f06b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:51.697930 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5451711-14fc-45c6-9892-f01b11a39638 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:51.701972 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:51.715146 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01df47a-ffb3-4aaa-abc4-021ed41c8978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:51.719787 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01df47a-ffb3-4aaa-abc4-021ed41c8978 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:51.720335 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01df47a-ffb3-4aaa-abc4-021ed41c8978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:51.720651 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01df47a-ffb3-4aaa-abc4-021ed41c8978 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:51.721528 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01df47a-ffb3-4aaa-abc4-021ed41c8978 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:51.723173 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01df47a-ffb3-4aaa-abc4-021ed41c8978 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:51.757096 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 826/1620] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:51 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:51.760895 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01df47a-ffb3-4aaa-abc4-021ed41c8978 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:51.791795 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01df47a-ffb3-4aaa-abc4-021ed41c8978 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:51.799419 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4bb6ea-6768-41ee-a2a8-a5f0731f06b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:51.803378 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4bb6ea-6768-41ee-a2a8-a5f0731f06b1 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:51.804089 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8423da0e-a44b-4bf6-a40c-445911df30b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:51.804826 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8423da0e-a44b-4bf6-a40c-445911df30b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:51.805112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8423da0e-a44b-4bf6-a40c-445911df30b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:51.805391 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8423da0e-a44b-4bf6-a40c-445911df30b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:51.826013 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 795/1621] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:51.827838 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8423da0e-a44b-4bf6-a40c-445911df30b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:51.829193 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2b327-449a-4b0e-a996-19320d5ed3c0 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:51.829485 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2b327-449a-4b0e-a996-19320d5ed3c0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:51.829763 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2b327-449a-4b0e-a996-19320d5ed3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:51.829955 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8423da0e-a44b-4bf6-a40c-445911df30b1 service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:51.850075 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 827/1622] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:51.850941 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8423da0e-a44b-4bf6-a40c-445911df30b1 service neutron] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:51.862644 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8423da0e-a44b-4bf6-a40c-445911df30b1 service neutron] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:51.869354 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f5226-193b-42b0-9832-f2c4b618294d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:51.870203 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f5226-193b-42b0-9832-f2c4b618294d admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:51.871069 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f5226-193b-42b0-9832-f2c4b618294d admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:51.871354 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f5226-193b-42b0-9832-f2c4b618294d admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:51.872028 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f5226-193b-42b0-9832-f2c4b618294d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:51.874666 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f5226-193b-42b0-9832-f2c4b618294d admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:51.930247 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-70c2b327-449a-4b0e-a996-19320d5ed3c0 None None] MFA Rules not processed for user `a246654c783b4cc68d212e91bb05d8c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:51.934292 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 828/1623] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:08:51 2022] POST /identity/v3/projects => generated 377 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:51.934440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-70c2b327-449a-4b0e-a996-19320d5ed3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:51.934547 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f5226-193b-42b0-9832-f2c4b618294d admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:52.035022 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:52.090814 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f5226-193b-42b0-9832-f2c4b618294d admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:52.101651 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ac833-491d-4ab8-b4be-42d82c5be037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:52.102919 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ac833-491d-4ab8-b4be-42d82c5be037 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:52.103429 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ac833-491d-4ab8-b4be-42d82c5be037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:52.103940 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ac833-491d-4ab8-b4be-42d82c5be037 admin admin] PATH_INFO: `/v3/projects/6451650d419347f8b215b80ba3cb4bd6/users/47f31c68f0be45d3b415c3afcb2f3eb0/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:52.109866 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ac833-491d-4ab8-b4be-42d82c5be037 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6451650d419347f8b215b80ba3cb4bd6, user_id=47f31c68f0be45d3b415c3afcb2f3eb0, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:52.112983 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ac833-491d-4ab8-b4be-42d82c5be037 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:52.122636 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 796/1624] 10.222.0.48 () {58 vars in 1059 bytes} [Mon Aug 1 14:08:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:52.125797 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c2b327-449a-4b0e-a996-19320d5ed3c0 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:52.145661 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 829/1625] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:51 2022] PUT /identity/v3/projects/6451650d419347f8b215b80ba3cb4bd6/users/47f31c68f0be45d3b415c3afcb2f3eb0/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:52.146880 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ac833-491d-4ab8-b4be-42d82c5be037 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:52.147126 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c2b327-449a-4b0e-a996-19320d5ed3c0 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:52.302577 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b879885a-b383-4234-9e6b-29647616309e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:52.303424 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879885a-b383-4234-9e6b-29647616309e admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:52.303720 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879885a-b383-4234-9e6b-29647616309e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:52.304004 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879885a-b383-4234-9e6b-29647616309e admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:52.304713 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b879885a-b383-4234-9e6b-29647616309e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:52.307496 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b879885a-b383-4234-9e6b-29647616309e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:52.332108 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ac833-491d-4ab8-b4be-42d82c5be037 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:52.344545 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1456e32b-9ccc-4082-a3d4-77b35664e710 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:52.345862 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456e32b-9ccc-4082-a3d4-77b35664e710 service nova] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:52.346370 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456e32b-9ccc-4082-a3d4-77b35664e710 service nova] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:52.346859 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456e32b-9ccc-4082-a3d4-77b35664e710 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:52.375873 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 797/1626] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:08:52 2022] POST /identity/v3/users => generated 295 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:08:52.375873 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b33503-224a-4b81-afc4-4526f3b8bdc5 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:52.375873 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b33503-224a-4b81-afc4-4526f3b8bdc5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:52.375873 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b33503-224a-4b81-afc4-4526f3b8bdc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:52.444846 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-50b33503-224a-4b81-afc4-4526f3b8bdc5 None None] MFA Rules not processed for user `47f31c68f0be45d3b415c3afcb2f3eb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:52.447642 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-50b33503-224a-4b81-afc4-4526f3b8bdc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:52.478737 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1456e32b-9ccc-4082-a3d4-77b35664e710 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:52.481140 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1456e32b-9ccc-4082-a3d4-77b35664e710 service nova] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:52.503474 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:52.530467 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 830/1627] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:52.531560 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1456e32b-9ccc-4082-a3d4-77b35664e710 service nova] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:52.586586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1456e32b-9ccc-4082-a3d4-77b35664e710 service nova] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:52.604414 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2fba6-3cba-4fb1-b1a9-9e776428e1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:52.605764 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2fba6-3cba-4fb1-b1a9-9e776428e1a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:52.606740 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2fba6-3cba-4fb1-b1a9-9e776428e1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:52.607638 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2fba6-3cba-4fb1-b1a9-9e776428e1a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:52.608364 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2fba6-3cba-4fb1-b1a9-9e776428e1a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:52.615846 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 798/1628] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:08:52 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:52.628517 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2fba6-3cba-4fb1-b1a9-9e776428e1a7 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:52.629632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b33503-224a-4b81-afc4-4526f3b8bdc5 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:52.660569 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 831/1629] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:52 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:08:52.677703 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2fba6-3cba-4fb1-b1a9-9e776428e1a7 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:52.704961 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2fba6-3cba-4fb1-b1a9-9e776428e1a7 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:52.733264 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609d29c-2a8b-4958-bfc4-24603be918c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:52.734386 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609d29c-2a8b-4958-bfc4-24603be918c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:52.734884 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609d29c-2a8b-4958-bfc4-24603be918c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:52.735967 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609d29c-2a8b-4958-bfc4-24603be918c5 admin admin] PATH_INFO: `/v3/projects/0f4160c3d97743fb91bcb541e5843ac9/users/7a49246b355d43c4ba5c241e19bde45e/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:52.808869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9609d29c-2a8b-4958-bfc4-24603be918c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f4160c3d97743fb91bcb541e5843ac9, user_id=7a49246b355d43c4ba5c241e19bde45e, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:52.818219 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b33503-224a-4b81-afc4-4526f3b8bdc5 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:52.822105 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9609d29c-2a8b-4958-bfc4-24603be918c5 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:52.837340 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af62b95-8882-43c7-ae0f-f4bfc28b73c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:52.838172 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af62b95-8882-43c7-ae0f-f4bfc28b73c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:52.838564 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af62b95-8882-43c7-ae0f-f4bfc28b73c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:52.839891 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af62b95-8882-43c7-ae0f-f4bfc28b73c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:52.857065 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 832/1630] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:52 2022] PUT /identity/v3/projects/0f4160c3d97743fb91bcb541e5843ac9/users/7a49246b355d43c4ba5c241e19bde45e/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:52.862778 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609d29c-2a8b-4958-bfc4-24603be918c5 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:52.873420 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af62b95-8882-43c7-ae0f-f4bfc28b73c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:52.875401 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af62b95-8882-43c7-ae0f-f4bfc28b73c2 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:52.913981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 799/1631] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:52 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:08:52.987655 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:53.042379 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19441136-3ed9-493c-b90c-59f075f82b6d None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:53.042956 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19441136-3ed9-493c-b90c-59f075f82b6d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:53.043388 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19441136-3ed9-493c-b90c-59f075f82b6d None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:53.046258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 800/1632] 10.222.0.48 () {54 vars in 912 bytes} [Mon Aug 1 14:08:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 14:08:53.054780 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258eaee0-897e-42b3-9a5d-a68b5ceba0e5 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:53.055075 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258eaee0-897e-42b3-9a5d-a68b5ceba0e5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:53.055710 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258eaee0-897e-42b3-9a5d-a68b5ceba0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:53.089686 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609d29c-2a8b-4958-bfc4-24603be918c5 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:53.106018 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-258eaee0-897e-42b3-9a5d-a68b5ceba0e5 None None] MFA Rules not processed for user `a246654c783b4cc68d212e91bb05d8c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:53.107491 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-258eaee0-897e-42b3-9a5d-a68b5ceba0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:53.108106 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b2955-9fab-44c5-bb4b-179ddcedb23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:53.108889 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b2955-9fab-44c5-bb4b-179ddcedb23c admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:53.109277 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b2955-9fab-44c5-bb4b-179ddcedb23c admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:53.109647 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b2955-9fab-44c5-bb4b-179ddcedb23c admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:53.111112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b2955-9fab-44c5-bb4b-179ddcedb23c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:53.112888 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b2955-9fab-44c5-bb4b-179ddcedb23c admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:53.138500 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 833/1633] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:08:52 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:08:53.138500 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b2955-9fab-44c5-bb4b-179ddcedb23c admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:53.244126 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 801/1634] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 14:08:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:08:53.245320 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258eaee0-897e-42b3-9a5d-a68b5ceba0e5 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:53.252323 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b2955-9fab-44c5-bb4b-179ddcedb23c admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:53.264110 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee08268d-3ab7-4b3b-804e-4d6aa304c0c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:53.265174 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08268d-3ab7-4b3b-804e-4d6aa304c0c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:53.265532 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08268d-3ab7-4b3b-804e-4d6aa304c0c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:53.265879 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08268d-3ab7-4b3b-804e-4d6aa304c0c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:53.281155 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258eaee0-897e-42b3-9a5d-a68b5ceba0e5 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:53.305586 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cb967b-1f73-4786-9d9b-d3eb90096d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:53.306636 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb967b-1f73-4786-9d9b-d3eb90096d0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:53.307077 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb967b-1f73-4786-9d9b-d3eb90096d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:53.307510 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb967b-1f73-4786-9d9b-d3eb90096d0f admin admin] PATH_INFO: `/v3/projects/0f4160c3d97743fb91bcb541e5843ac9/users/7a49246b355d43c4ba5c241e19bde45e/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:53.317600 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cb967b-1f73-4786-9d9b-d3eb90096d0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f4160c3d97743fb91bcb541e5843ac9, user_id=7a49246b355d43c4ba5c241e19bde45e, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:53.326678 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cb967b-1f73-4786-9d9b-d3eb90096d0f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:53.358707 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 802/1635] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:08:53 2022] PUT /identity/v3/projects/0f4160c3d97743fb91bcb541e5843ac9/users/7a49246b355d43c4ba5c241e19bde45e/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:08:53.359738 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cb967b-1f73-4786-9d9b-d3eb90096d0f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:53.375395 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee08268d-3ab7-4b3b-804e-4d6aa304c0c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:53.380621 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee08268d-3ab7-4b3b-804e-4d6aa304c0c4 service cinder] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:53.410895 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 834/1636] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:53.416620 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c49b6c-218a-4ba1-8b6d-3ae6eb6b94a3 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:53.416620 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c49b6c-218a-4ba1-8b6d-3ae6eb6b94a3 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:53.416832 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c49b6c-218a-4ba1-8b6d-3ae6eb6b94a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:53.428961 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:53.466248 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cb967b-1f73-4786-9d9b-d3eb90096d0f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:53.471846 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2769e5-0c8c-42e3-b29d-eadabeb43b20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:53.473722 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2769e5-0c8c-42e3-b29d-eadabeb43b20 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:53.474773 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2769e5-0c8c-42e3-b29d-eadabeb43b20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:53.475462 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2769e5-0c8c-42e3-b29d-eadabeb43b20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:53.497102 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-98c49b6c-218a-4ba1-8b6d-3ae6eb6b94a3 None None] MFA Rules not processed for user `7a49246b355d43c4ba5c241e19bde45e`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:53.500227 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-98c49b6c-218a-4ba1-8b6d-3ae6eb6b94a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:53.638297 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2769e5-0c8c-42e3-b29d-eadabeb43b20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:53.644920 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2769e5-0c8c-42e3-b29d-eadabeb43b20 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:53.688648 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 803/1637] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:53.722744 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 835/1638] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:08:53 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:53.789053 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2769e5-0c8c-42e3-b29d-eadabeb43b20 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:53.808006 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2769e5-0c8c-42e3-b29d-eadabeb43b20 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:53.824568 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb62890-4cd2-4f3c-addb-5e80e390d687 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:53.825265 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb62890-4cd2-4f3c-addb-5e80e390d687 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:53.825545 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb62890-4cd2-4f3c-addb-5e80e390d687 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:53.825817 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb62890-4cd2-4f3c-addb-5e80e390d687 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:54.012808 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb62890-4cd2-4f3c-addb-5e80e390d687 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:54.016399 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb62890-4cd2-4f3c-addb-5e80e390d687 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:54.044580 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 804/1639] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:08:55.294825 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd3566-9810-4634-947b-24deaf96210d None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:55.295444 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd3566-9810-4634-947b-24deaf96210d None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:55.295899 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd3566-9810-4634-947b-24deaf96210d None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:55.297739 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 836/1640] 10.222.0.48 () {54 vars in 962 bytes} [Mon Aug 1 14:08:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:08:55.319814 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10df1a8-1fd9-4a4e-8ef3-916e88234bfd None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:55.320425 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10df1a8-1fd9-4a4e-8ef3-916e88234bfd None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:55.320851 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10df1a8-1fd9-4a4e-8ef3-916e88234bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:55.362870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b10df1a8-1fd9-4a4e-8ef3-916e88234bfd None None] MFA Rules not processed for user `a246654c783b4cc68d212e91bb05d8c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:55.364320 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b10df1a8-1fd9-4a4e-8ef3-916e88234bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:55.387901 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 805/1641] 10.222.0.48 () {58 vars in 1059 bytes} [Mon Aug 1 14:08:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:55.393583 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d80e4-f232-4ff0-9359-853a27237cf4 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:55.394014 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d80e4-f232-4ff0-9359-853a27237cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:55.394561 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d80e4-f232-4ff0-9359-853a27237cf4 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:55.396156 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 837/1642] 10.222.0.48 () {54 vars in 912 bytes} [Mon Aug 1 14:08:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:08:55.402243 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801acbe-ad5f-4765-be8b-afeac0a67f91 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:55.402961 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801acbe-ad5f-4765-be8b-afeac0a67f91 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:55.403467 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801acbe-ad5f-4765-be8b-afeac0a67f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:55.441230 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f801acbe-ad5f-4765-be8b-afeac0a67f91 None None] MFA Rules not processed for user `a246654c783b4cc68d212e91bb05d8c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:08:55.442793 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f801acbe-ad5f-4765-be8b-afeac0a67f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:08:55.473377 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 806/1643] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 14:08:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:08:55.481520 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d80e4-f232-4ff0-9359-853a27237cf4 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:08:55.490228 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:08:55.502141 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d80e4-f232-4ff0-9359-853a27237cf4 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:08:55.506428 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb1438-081c-4307-943a-3743e4bd4ef0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:08:55.507367 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb1438-081c-4307-943a-3743e4bd4ef0 service cinder] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:08:55.508653 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb1438-081c-4307-943a-3743e4bd4ef0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:08:55.509153 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb1438-081c-4307-943a-3743e4bd4ef0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:08:55.540603 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb1438-081c-4307-943a-3743e4bd4ef0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:08:55.542595 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb1438-081c-4307-943a-3743e4bd4ef0 service cinder] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:08:55.574773 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 838/1644] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:08:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:02.321121 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28e707-e1bf-4ef8-b6c6-a5445f321fe8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:02.321724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28e707-e1bf-4ef8-b6c6-a5445f321fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:02.321820 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28e707-e1bf-4ef8-b6c6-a5445f321fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:02.422052 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-9f28e707-e1bf-4ef8-b6c6-a5445f321fe8 None None] MFA Rules not processed for user `822ea58eeb4e4d29b132964c0e3355db`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:02.423453 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9f28e707-e1bf-4ef8-b6c6-a5445f321fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:02.598908 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:02.626819 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 807/1645] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:02.639936 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420e716-7e67-430c-8dd7-cd380a3e69de None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:02.640872 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420e716-7e67-430c-8dd7-cd380a3e69de None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:02.641291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420e716-7e67-430c-8dd7-cd380a3e69de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:02.690917 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-7420e716-7e67-430c-8dd7-cd380a3e69de None None] MFA Rules not processed for user `822ea58eeb4e4d29b132964c0e3355db`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:02.692188 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-7420e716-7e67-430c-8dd7-cd380a3e69de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:02.709976 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 839/1646] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:02.723417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f28e707-e1bf-4ef8-b6c6-a5445f321fe8 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:02.745587 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f28e707-e1bf-4ef8-b6c6-a5445f321fe8 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:02.770388 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a345b7ec-9cca-4e76-b2d8-17e126f05546 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:02.770795 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345b7ec-9cca-4e76-b2d8-17e126f05546 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:02.771088 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345b7ec-9cca-4e76-b2d8-17e126f05546 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:02.771883 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345b7ec-9cca-4e76-b2d8-17e126f05546 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:02.798587 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a345b7ec-9cca-4e76-b2d8-17e126f05546 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:02.800114 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a345b7ec-9cca-4e76-b2d8-17e126f05546 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:02.860930 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 808/1647] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:02 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:03.597575 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0094a-3e2e-4556-9366-4a063a425762 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:03.598419 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0094a-3e2e-4556-9366-4a063a425762 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:03.598419 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0094a-3e2e-4556-9366-4a063a425762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:03.654258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e70dd-3941-48a1-8448-ee4631555154 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:03.654735 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e70dd-3941-48a1-8448-ee4631555154 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:03.654821 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e70dd-3941-48a1-8448-ee4631555154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:03.687698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-a7c0094a-3e2e-4556-9366-4a063a425762 None None] MFA Rules not processed for user `f379b005418a4e399c3b72cbbef6e00d`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:03.690007 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-a7c0094a-3e2e-4556-9366-4a063a425762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:03.734963 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-453e70dd-3941-48a1-8448-ee4631555154 None None] MFA Rules not processed for user `ca3cd1c2aec94b43aff0034b812a069d`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:03.740640 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-453e70dd-3941-48a1-8448-ee4631555154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:03.833119 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 809/1648] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:03.851683 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e70dd-3941-48a1-8448-ee4631555154 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:03.878956 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e70dd-3941-48a1-8448-ee4631555154 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:03.882696 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 840/1649] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:03.884386 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80f1f5-e768-4216-8f71-cf3eb474b9de None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:03.884908 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80f1f5-e768-4216-8f71-cf3eb474b9de None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:03.885492 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80f1f5-e768-4216-8f71-cf3eb474b9de None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:03.888439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 841/1650] 10.222.0.48 () {54 vars in 912 bytes} [Mon Aug 1 14:09:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:09:03.892634 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd4423-f7d7-4916-b310-f2930802b3e8 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:03.893525 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd4423-f7d7-4916-b310-f2930802b3e8 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:03.894804 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd4423-f7d7-4916-b310-f2930802b3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:03.899991 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7b760-25b9-425f-9ca8-057a7661fbee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:03.901207 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7b760-25b9-425f-9ca8-057a7661fbee service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:03.901497 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7b760-25b9-425f-9ca8-057a7661fbee service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:03.901778 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7b760-25b9-425f-9ca8-057a7661fbee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:03.918392 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c7b760-25b9-425f-9ca8-057a7661fbee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:03.919915 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c7b760-25b9-425f-9ca8-057a7661fbee service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:03.943593 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 810/1651] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:03.945123 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e6ed1-30fe-4c00-b94b-964203e1ba00 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:03.945434 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e6ed1-30fe-4c00-b94b-964203e1ba00 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:03.945791 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e6ed1-30fe-4c00-b94b-964203e1ba00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:03.950342 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-1ccd4423-f7d7-4916-b310-f2930802b3e8 None None] MFA Rules not processed for user `f379b005418a4e399c3b72cbbef6e00d`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:03.952044 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-1ccd4423-f7d7-4916-b310-f2930802b3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:03.971728 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 842/1652] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:03.995660 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccd4423-f7d7-4916-b310-f2930802b3e8 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:04.009468 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccd4423-f7d7-4916-b310-f2930802b3e8 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:04.021928 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c3210-e7de-426c-a731-467b8dd45900 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:04.023074 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c3210-e7de-426c-a731-467b8dd45900 service cinder] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:04.023655 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c3210-e7de-426c-a731-467b8dd45900 service cinder] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:04.024156 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c3210-e7de-426c-a731-467b8dd45900 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:04.034146 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-822e6ed1-30fe-4c00-b94b-964203e1ba00 None None] MFA Rules not processed for user `fb9f698e09d246bcbea855613764fbe1`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:04.035705 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-822e6ed1-30fe-4c00-b94b-964203e1ba00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:04.039567 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c3210-e7de-426c-a731-467b8dd45900 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:04.041078 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c3210-e7de-426c-a731-467b8dd45900 service cinder] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:04.054790 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 843/1653] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:03 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:09:04.154617 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 811/1654] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 14:09:03 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:04.169112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c3210-e7de-426c-a731-467b8dd45900 service cinder] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:04.185831 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c3210-e7de-426c-a731-467b8dd45900 service cinder] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:04.213839 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701f83bd-0b06-45ef-962f-6728d0c6c6e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:04.220458 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f83bd-0b06-45ef-962f-6728d0c6c6e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:04.220827 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f83bd-0b06-45ef-962f-6728d0c6c6e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:04.221447 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f83bd-0b06-45ef-962f-6728d0c6c6e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:04.248589 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822e6ed1-30fe-4c00-b94b-964203e1ba00 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:04.254090 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701f83bd-0b06-45ef-962f-6728d0c6c6e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:04.255769 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701f83bd-0b06-45ef-962f-6728d0c6c6e4 service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:04.266257 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 844/1655] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:04.281475 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822e6ed1-30fe-4c00-b94b-964203e1ba00 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:04.300001 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22833ab-909a-4de5-952b-44d3941e0bc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:04.300688 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22833ab-909a-4de5-952b-44d3941e0bc1 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:04.300973 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22833ab-909a-4de5-952b-44d3941e0bc1 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:04.301251 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22833ab-909a-4de5-952b-44d3941e0bc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:04.321842 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22833ab-909a-4de5-952b-44d3941e0bc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:04.323967 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22833ab-909a-4de5-952b-44d3941e0bc1 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:04.356252 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 812/1656] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:04 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:05.514546 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878776c-d17b-4581-9298-22fe42676dd1 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:05.516161 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878776c-d17b-4581-9298-22fe42676dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:05.516455 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878776c-d17b-4581-9298-22fe42676dd1 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:05.517943 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 845/1657] 10.222.0.48 () {54 vars in 961 bytes} [Mon Aug 1 14:09:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:09:05.529899 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089801e-9063-4b54-a9bc-bb331ed288f8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:05.530449 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089801e-9063-4b54-a9bc-bb331ed288f8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:05.530823 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089801e-9063-4b54-a9bc-bb331ed288f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:05.562206 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d089801e-9063-4b54-a9bc-bb331ed288f8 None None] MFA Rules not processed for user `e294fe796a1a4ac89041a532856ad627`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:05.564914 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d089801e-9063-4b54-a9bc-bb331ed288f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:05.695347 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:05.781850 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 813/1658] 10.222.0.48 () {58 vars in 1058 bytes} [Mon Aug 1 14:09:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:05.789482 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0878776c-d17b-4581-9298-22fe42676dd1 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:05.828977 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0878776c-d17b-4581-9298-22fe42676dd1 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:05.837293 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d3b3ad-cfc1-480c-b836-476d598b9c3f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e294fe796a1a4ac89041a532856ad627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:05.839629 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d3b3ad-cfc1-480c-b836-476d598b9c3f service placement] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:05.840086 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d3b3ad-cfc1-480c-b836-476d598b9c3f service placement] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:05.840539 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d3b3ad-cfc1-480c-b836-476d598b9c3f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:05.843852 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-87d3b3ad-cfc1-480c-b836-476d598b9c3f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:05.865651 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d3b3ad-cfc1-480c-b836-476d598b9c3f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:05.867575 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d3b3ad-cfc1-480c-b836-476d598b9c3f service placement] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:05.884532 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 846/1659] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:06.208761 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba72edf-e45a-46f7-8cc0-ff0556c74f86 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:06.209438 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba72edf-e45a-46f7-8cc0-ff0556c74f86 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:06.209863 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba72edf-e45a-46f7-8cc0-ff0556c74f86 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:06.212052 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 814/1660] 10.222.0.48 () {54 vars in 912 bytes} [Mon Aug 1 14:09:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:09:06.226107 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b97d505-96d8-42dc-a730-fe36ed580e4e None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:06.226664 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b97d505-96d8-42dc-a730-fe36ed580e4e None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:06.227306 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b97d505-96d8-42dc-a730-fe36ed580e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:06.266205 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-2b97d505-96d8-42dc-a730-fe36ed580e4e None None] MFA Rules not processed for user `fb9f698e09d246bcbea855613764fbe1`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:06.270573 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-2b97d505-96d8-42dc-a730-fe36ed580e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:06.287205 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 847/1661] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 14:09:06 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:06.337749 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba72edf-e45a-46f7-8cc0-ff0556c74f86 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:06.365526 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba72edf-e45a-46f7-8cc0-ff0556c74f86 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:06.380785 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba0d3b-064b-4516-924d-9ddec5435ab4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:06.383913 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba0d3b-064b-4516-924d-9ddec5435ab4 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:06.384927 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba0d3b-064b-4516-924d-9ddec5435ab4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:06.385615 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba0d3b-064b-4516-924d-9ddec5435ab4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:06.420198 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba0d3b-064b-4516-924d-9ddec5435ab4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:06.422267 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba0d3b-064b-4516-924d-9ddec5435ab4 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:06.448662 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 815/1662] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:07.660990 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70881c-a921-4471-adf2-9c5301c6befe None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:07.661833 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70881c-a921-4471-adf2-9c5301c6befe None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:07.662268 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70881c-a921-4471-adf2-9c5301c6befe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:07.712440 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-6c70881c-a921-4471-adf2-9c5301c6befe None None] MFA Rules not processed for user `62cc969defd64837a20e9a7dbda5b4fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:07.714635 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-6c70881c-a921-4471-adf2-9c5301c6befe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:07.813045 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 848/1663] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:07.830870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba0d3b-064b-4516-924d-9ddec5435ab4 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:07.857654 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba0d3b-064b-4516-924d-9ddec5435ab4 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:07.867560 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43324704-7ff2-4e82-bde2-af2abff7c1c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:07.868059 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43324704-7ff2-4e82-bde2-af2abff7c1c4 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:07.868466 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43324704-7ff2-4e82-bde2-af2abff7c1c4 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:07.868936 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43324704-7ff2-4e82-bde2-af2abff7c1c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:07.869421 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643fe5f-fb6e-4af1-956d-e0965de6cf39 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:07.869803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643fe5f-fb6e-4af1-956d-e0965de6cf39 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:07.870150 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643fe5f-fb6e-4af1-956d-e0965de6cf39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:07.902663 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-e643fe5f-fb6e-4af1-956d-e0965de6cf39 None None] MFA Rules not processed for user `f379b005418a4e399c3b72cbbef6e00d`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:07.904292 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-e643fe5f-fb6e-4af1-956d-e0965de6cf39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:07.911592 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43324704-7ff2-4e82-bde2-af2abff7c1c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:07.913908 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 849/1664] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:07.915478 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43324704-7ff2-4e82-bde2-af2abff7c1c4 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:07.950213 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 816/1665] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:08.862674 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd80ec-ed59-4859-b141-99155d5432e3 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:08.863827 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd80ec-ed59-4859-b141-99155d5432e3 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:08.864268 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd80ec-ed59-4859-b141-99155d5432e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:08.877318 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:08.905197 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-6ffd80ec-ed59-4859-b141-99155d5432e3 None None] MFA Rules not processed for user `ca3cd1c2aec94b43aff0034b812a069d`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:08.910252 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-6ffd80ec-ed59-4859-b141-99155d5432e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:08.937341 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 850/1666] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:11.538416 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179a49f-069b-40e9-a199-e7eda8f4d060 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:11.539252 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179a49f-069b-40e9-a199-e7eda8f4d060 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:11.540136 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179a49f-069b-40e9-a199-e7eda8f4d060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:11.582232 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4179a49f-069b-40e9-a199-e7eda8f4d060 None None] MFA Rules not processed for user `62cc969defd64837a20e9a7dbda5b4fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:11.583728 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4179a49f-069b-40e9-a199-e7eda8f4d060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:11.598796 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 817/1667] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:14.096847 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852bb1b7-98b4-49e0-920d-05e50850d2b5 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:14.097438 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852bb1b7-98b4-49e0-920d-05e50850d2b5 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:14.097531 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852bb1b7-98b4-49e0-920d-05e50850d2b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:14.104679 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bf48f-289b-4841-83da-011e773562ee None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:14.105227 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bf48f-289b-4841-83da-011e773562ee None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:14.105227 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bf48f-289b-4841-83da-011e773562ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:14.130547 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-852bb1b7-98b4-49e0-920d-05e50850d2b5 None None] MFA Rules not processed for user `fb9f698e09d246bcbea855613764fbe1`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:14.132024 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-852bb1b7-98b4-49e0-920d-05e50850d2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:14.142822 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 851/1668] 10.222.0.71 () {58 vars in 1015 bytes} [Mon Aug 1 14:09:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:14.157146 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852bb1b7-98b4-49e0-920d-05e50850d2b5 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:14.173268 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852bb1b7-98b4-49e0-920d-05e50850d2b5 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:14.183104 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21dceea-53b3-48c3-8102-3865f0039fe8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:14.184263 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21dceea-53b3-48c3-8102-3865f0039fe8 service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:14.184902 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21dceea-53b3-48c3-8102-3865f0039fe8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:14.185406 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21dceea-53b3-48c3-8102-3865f0039fe8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:14.210200 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21dceea-53b3-48c3-8102-3865f0039fe8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:14.212233 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21dceea-53b3-48c3-8102-3865f0039fe8 service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:14.238928 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 852/1669] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:14.268565 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-944bf48f-289b-4841-83da-011e773562ee None None] MFA Rules not processed for user `5007a2f8e6244b0788f0a554cc572fe5`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:14.270823 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-944bf48f-289b-4841-83da-011e773562ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:14.494857 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 818/1670] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 14:09:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:14.926138 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21dceea-53b3-48c3-8102-3865f0039fe8 service neutron] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:14.932731 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-b21dceea-53b3-48c3-8102-3865f0039fe8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:15.222989 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21dceea-53b3-48c3-8102-3865f0039fe8 service neutron] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:15.232340 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572ac6a-cf72-4d47-a697-a33a1fd2a89d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2a6d7bd10435ba2c8e552d808b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:15.233028 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572ac6a-cf72-4d47-a697-a33a1fd2a89d service swift] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:15.233316 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572ac6a-cf72-4d47-a697-a33a1fd2a89d service swift] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:15.233591 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572ac6a-cf72-4d47-a697-a33a1fd2a89d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:15.264098 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9572ac6a-cf72-4d47-a697-a33a1fd2a89d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:15.266118 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9572ac6a-cf72-4d47-a697-a33a1fd2a89d service swift] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:15.282982 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 853/1671] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 14:09:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:09:16.231281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944bf48f-289b-4841-83da-011e773562ee None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:16.365937 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944bf48f-289b-4841-83da-011e773562ee None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:16.372019 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e0c50-b12f-4280-853e-36862faa5eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:16.373031 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e0c50-b12f-4280-853e-36862faa5eca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:16.373545 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e0c50-b12f-4280-853e-36862faa5eca admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:16.374038 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e0c50-b12f-4280-853e-36862faa5eca admin admin] PATH_INFO: `/v3/users/9c968d77125448a7860a47707224e1a4` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:16.375632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696e0c50-b12f-4280-853e-36862faa5eca admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c968d77125448a7860a47707224e1a4)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:16.377039 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: WARNING py.warnings [None req-696e0c50-b12f-4280-853e-36862faa5eca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:09:16.377039 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: warnings.warn(msg) Aug 01 14:09:16.377039 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: Aug 01 14:09:16.379119 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696e0c50-b12f-4280-853e-36862faa5eca admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:16.460653 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:16.553642 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-696e0c50-b12f-4280-853e-36862faa5eca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c968d77125448a7860a47707224e1a4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:16.600720 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-696e0c50-b12f-4280-853e-36862faa5eca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c968d77125448a7860a47707224e1a4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:16.607250 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-696e0c50-b12f-4280-853e-36862faa5eca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c968d77125448a7860a47707224e1a4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:16.625040 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-696e0c50-b12f-4280-853e-36862faa5eca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c968d77125448a7860a47707224e1a4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:16.625918 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-696e0c50-b12f-4280-853e-36862faa5eca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c968d77125448a7860a47707224e1a4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:16.673693 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 819/1672] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:16 2022] DELETE /identity/v3/users/9c968d77125448a7860a47707224e1a4 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:16.678358 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e48b2-66b8-4e78-9180-b15c2b3ea354 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:16.679331 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e48b2-66b8-4e78-9180-b15c2b3ea354 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:16.679864 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e48b2-66b8-4e78-9180-b15c2b3ea354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:16.706586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-1e3e48b2-66b8-4e78-9180-b15c2b3ea354 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:16.708225 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-1e3e48b2-66b8-4e78-9180-b15c2b3ea354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:16.862362 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 854/1673] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:16.879091 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e0c50-b12f-4280-853e-36862faa5eca admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:16.882045 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-696e0c50-b12f-4280-853e-36862faa5eca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:16.955084 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e48b2-66b8-4e78-9180-b15c2b3ea354 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:16.957146 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-1e3e48b2-66b8-4e78-9180-b15c2b3ea354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:16.974960 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e48b2-66b8-4e78-9180-b15c2b3ea354 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:16.977488 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e0c50-b12f-4280-853e-36862faa5eca admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:16.984292 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625f0228-009e-4a94-93cf-acaf08902736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:16.985316 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf26e36-0f4a-4c8b-bb63-723e2903156e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:16.987187 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f0228-009e-4a94-93cf-acaf08902736 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:16.987762 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf26e36-0f4a-4c8b-bb63-723e2903156e service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:16.988136 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f0228-009e-4a94-93cf-acaf08902736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:16.988503 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf26e36-0f4a-4c8b-bb63-723e2903156e service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:16.988870 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f0228-009e-4a94-93cf-acaf08902736 admin admin] PATH_INFO: `/v3/users/73f955986c3b4d77a6f5cd1a823155a8` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:16.996672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625f0228-009e-4a94-93cf-acaf08902736 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73f955986c3b4d77a6f5cd1a823155a8)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:16.998641 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625f0228-009e-4a94-93cf-acaf08902736 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:17.000292 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf26e36-0f4a-4c8b-bb63-723e2903156e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:17.021313 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf26e36-0f4a-4c8b-bb63-723e2903156e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:17.023768 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf26e36-0f4a-4c8b-bb63-723e2903156e service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:17.046522 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 820/1674] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:17.114075 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:17.120534 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-625f0228-009e-4a94-93cf-acaf08902736 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73f955986c3b4d77a6f5cd1a823155a8'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:17.122662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-625f0228-009e-4a94-93cf-acaf08902736 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73f955986c3b4d77a6f5cd1a823155a8'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:17.154457 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5f092-d2fc-44dc-bef3-48ab25b26329 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:17.154587 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-625f0228-009e-4a94-93cf-acaf08902736 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73f955986c3b4d77a6f5cd1a823155a8'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:17.155311 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5f092-d2fc-44dc-bef3-48ab25b26329 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:17.155764 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5f092-d2fc-44dc-bef3-48ab25b26329 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:17.157528 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 821/1675] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 14:09:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:09:17.167759 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-625f0228-009e-4a94-93cf-acaf08902736 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73f955986c3b4d77a6f5cd1a823155a8'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:17.169450 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-625f0228-009e-4a94-93cf-acaf08902736 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73f955986c3b4d77a6f5cd1a823155a8'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:17.188712 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db5ed8-5d07-4856-af1a-0269a94fd6c3 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:17.189347 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db5ed8-5d07-4856-af1a-0269a94fd6c3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:17.189839 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db5ed8-5d07-4856-af1a-0269a94fd6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:17.229626 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-39db5ed8-5d07-4856-af1a-0269a94fd6c3 None None] MFA Rules not processed for user `a246654c783b4cc68d212e91bb05d8c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:17.231370 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-39db5ed8-5d07-4856-af1a-0269a94fd6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:17.261875 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 855/1676] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:16 2022] DELETE /identity/v3/users/73f955986c3b4d77a6f5cd1a823155a8 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:17.270575 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b6a08-dfb7-4ed0-8328-489230769c30 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:17.270877 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b6a08-dfb7-4ed0-8328-489230769c30 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:17.271163 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b6a08-dfb7-4ed0-8328-489230769c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:17.346224 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-222b6a08-dfb7-4ed0-8328-489230769c30 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:17.347784 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-222b6a08-dfb7-4ed0-8328-489230769c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:17.428588 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 822/1677] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 14:09:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:17.511952 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 856/1678] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:17.539494 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db5ed8-5d07-4856-af1a-0269a94fd6c3 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:17.541093 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-39db5ed8-5d07-4856-af1a-0269a94fd6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:17.557831 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db5ed8-5d07-4856-af1a-0269a94fd6c3 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:17.606682 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:17.651489 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4416303c-1e7f-4420-9862-cb22a3e79aa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:17.652132 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416303c-1e7f-4420-9862-cb22a3e79aa8 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:17.652418 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416303c-1e7f-4420-9862-cb22a3e79aa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:17.653229 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416303c-1e7f-4420-9862-cb22a3e79aa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:17.664868 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4416303c-1e7f-4420-9862-cb22a3e79aa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:17.666330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4416303c-1e7f-4420-9862-cb22a3e79aa8 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:17.679984 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 823/1679] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:17.746951 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222b6a08-dfb7-4ed0-8328-489230769c30 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:17.748834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-222b6a08-dfb7-4ed0-8328-489230769c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:17.838148 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4416303c-1e7f-4420-9862-cb22a3e79aa8 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:17.840956 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4416303c-1e7f-4420-9862-cb22a3e79aa8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:17.857328 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4416303c-1e7f-4420-9862-cb22a3e79aa8 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:17.865558 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e074a252-4835-44b3-9b13-9da9e80208be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:17.867160 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074a252-4835-44b3-9b13-9da9e80208be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:17.867910 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074a252-4835-44b3-9b13-9da9e80208be admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:17.868898 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074a252-4835-44b3-9b13-9da9e80208be admin admin] PATH_INFO: `/v3/projects/20a0f7842ef749f38d4bdf536f75d993` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:17.870700 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e074a252-4835-44b3-9b13-9da9e80208be admin admin] RBAC: Authorizing `identity:delete_project(project_id=20a0f7842ef749f38d4bdf536f75d993)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:17.872862 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e074a252-4835-44b3-9b13-9da9e80208be admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:17.909317 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222b6a08-dfb7-4ed0-8328-489230769c30 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:17.940910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c619c77-8f95-4e30-8343-2df23edefe98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:17.941665 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c619c77-8f95-4e30-8343-2df23edefe98 service nova] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:17.942057 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c619c77-8f95-4e30-8343-2df23edefe98 service nova] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:17.942411 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c619c77-8f95-4e30-8343-2df23edefe98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:17.953612 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c619c77-8f95-4e30-8343-2df23edefe98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:17.955408 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c619c77-8f95-4e30-8343-2df23edefe98 service nova] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:17.961174 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e074a252-4835-44b3-9b13-9da9e80208be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20a0f7842ef749f38d4bdf536f75d993', 'request_id': 'req-e074a252-4835-44b3-9b13-9da9e80208be'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:17.967539 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 857/1680] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:17.970876 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e074a252-4835-44b3-9b13-9da9e80208be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20a0f7842ef749f38d4bdf536f75d993', 'request_id': 'req-e074a252-4835-44b3-9b13-9da9e80208be'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:17.980333 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e074a252-4835-44b3-9b13-9da9e80208be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20a0f7842ef749f38d4bdf536f75d993', 'request_id': 'req-e074a252-4835-44b3-9b13-9da9e80208be'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:17.994968 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e074a252-4835-44b3-9b13-9da9e80208be admin admin] The token cache is being invalidate because project 20a0f7842ef749f38d4bdf536f75d993 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:17.995251 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e074a252-4835-44b3-9b13-9da9e80208be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:17.995997 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e074a252-4835-44b3-9b13-9da9e80208be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:17.997718 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 824/1681] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:17 2022] DELETE /identity/v3/projects/20a0f7842ef749f38d4bdf536f75d993 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:18.002941 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7512aa5-76aa-4728-ba79-ade1da1aecee None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:18.003234 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7512aa5-76aa-4728-ba79-ade1da1aecee None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:18.003537 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7512aa5-76aa-4728-ba79-ade1da1aecee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:18.024805 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-a7512aa5-76aa-4728-ba79-ade1da1aecee None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:18.026030 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-a7512aa5-76aa-4728-ba79-ade1da1aecee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:18.074504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:18.094228 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 858/1682] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:18.102191 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35a2d0-4ee8-4cfa-be56-5bcf4b9a77b3 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:18.102602 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35a2d0-4ee8-4cfa-be56-5bcf4b9a77b3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:18.103000 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35a2d0-4ee8-4cfa-be56-5bcf4b9a77b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:18.124619 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8c35a2d0-4ee8-4cfa-be56-5bcf4b9a77b3 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:18.126039 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8c35a2d0-4ee8-4cfa-be56-5bcf4b9a77b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:18.135314 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 825/1683] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:18.139941 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7512aa5-76aa-4728-ba79-ade1da1aecee None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:18.149940 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7512aa5-76aa-4728-ba79-ade1da1aecee None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:18.154510 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e5455e-40ef-4d7b-aa5b-14df41f9e348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:18.155127 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5455e-40ef-4d7b-aa5b-14df41f9e348 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:18.155414 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5455e-40ef-4d7b-aa5b-14df41f9e348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:18.155697 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5455e-40ef-4d7b-aa5b-14df41f9e348 admin admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:18.156576 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e5455e-40ef-4d7b-aa5b-14df41f9e348 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:18.156894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e5455e-40ef-4d7b-aa5b-14df41f9e348 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:18.158680 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e5455e-40ef-4d7b-aa5b-14df41f9e348 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:18.171336 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 859/1684] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:09:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:18.176329 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c35a2d0-4ee8-4cfa-be56-5bcf4b9a77b3 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:18.191905 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c35a2d0-4ee8-4cfa-be56-5bcf4b9a77b3 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:18.198646 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a10d3-dc01-4aef-a72b-88f35926e5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:18.199443 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a10d3-dc01-4aef-a72b-88f35926e5e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:18.199818 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a10d3-dc01-4aef-a72b-88f35926e5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:18.200171 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a10d3-dc01-4aef-a72b-88f35926e5e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:18.201383 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590a10d3-dc01-4aef-a72b-88f35926e5e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:18.204474 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590a10d3-dc01-4aef-a72b-88f35926e5e7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:18.418172 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e5455e-40ef-4d7b-aa5b-14df41f9e348 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:18.421437 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 826/1685] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:18 2022] POST /identity/v3/projects => generated 373 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:18.423195 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a10d3-dc01-4aef-a72b-88f35926e5e7 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:18.423831 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-c3e5455e-40ef-4d7b-aa5b-14df41f9e348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:18.439516 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e5455e-40ef-4d7b-aa5b-14df41f9e348 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:18.510502 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a10d3-dc01-4aef-a72b-88f35926e5e7 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:18.516263 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e200b-29ce-442d-a0bf-66a33dfa725e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:18.516953 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e200b-29ce-442d-a0bf-66a33dfa725e admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:18.517234 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e200b-29ce-442d-a0bf-66a33dfa725e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:18.517505 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e200b-29ce-442d-a0bf-66a33dfa725e admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:18.518206 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6e200b-29ce-442d-a0bf-66a33dfa725e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:18.518695 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08eede-20be-4b91-b9be-8020528e8dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:18.519386 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08eede-20be-4b91-b9be-8020528e8dcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:18.519678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08eede-20be-4b91-b9be-8020528e8dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:18.519957 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08eede-20be-4b91-b9be-8020528e8dcd admin admin] PATH_INFO: `/v3/projects/0993d5226b264f3fa129a55e74f0eb5a` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:18.520118 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6e200b-29ce-442d-a0bf-66a33dfa725e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:18.521230 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc08eede-20be-4b91-b9be-8020528e8dcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0993d5226b264f3fa129a55e74f0eb5a)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:18.522254 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: WARNING py.warnings [None req-fc08eede-20be-4b91-b9be-8020528e8dcd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 14:09:18.522254 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: warnings.warn(msg) Aug 01 14:09:18.522254 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: Aug 01 14:09:18.523504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc08eede-20be-4b91-b9be-8020528e8dcd admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:18.549874 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 827/1686] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:18 2022] POST /identity/v3/users => generated 287 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:18.553481 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e200b-29ce-442d-a0bf-66a33dfa725e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:18.563876 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e200b-29ce-442d-a0bf-66a33dfa725e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:18.568807 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ba4e8-b1f3-478b-994f-b865cae21398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:18.569163 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ba4e8-b1f3-478b-994f-b865cae21398 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:18.569692 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ba4e8-b1f3-478b-994f-b865cae21398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:18.570013 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ba4e8-b1f3-478b-994f-b865cae21398 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:18.570763 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ba4e8-b1f3-478b-994f-b865cae21398 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:18.572755 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ba4e8-b1f3-478b-994f-b865cae21398 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:18.587444 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 828/1687] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:18.588784 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ba4e8-b1f3-478b-994f-b865cae21398 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:18.599047 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ba4e8-b1f3-478b-994f-b865cae21398 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:18.601478 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-fc08eede-20be-4b91-b9be-8020528e8dcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0993d5226b264f3fa129a55e74f0eb5a', 'request_id': 'req-fc08eede-20be-4b91-b9be-8020528e8dcd'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:18.604052 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:18.620782 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-fc08eede-20be-4b91-b9be-8020528e8dcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0993d5226b264f3fa129a55e74f0eb5a', 'request_id': 'req-fc08eede-20be-4b91-b9be-8020528e8dcd'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:18.627506 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-fc08eede-20be-4b91-b9be-8020528e8dcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0993d5226b264f3fa129a55e74f0eb5a', 'request_id': 'req-fc08eede-20be-4b91-b9be-8020528e8dcd'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:18.641306 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-fc08eede-20be-4b91-b9be-8020528e8dcd admin admin] The token cache is being invalidate because project 0993d5226b264f3fa129a55e74f0eb5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:18.641597 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-fc08eede-20be-4b91-b9be-8020528e8dcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:18.642223 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-fc08eede-20be-4b91-b9be-8020528e8dcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:18.643824 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 860/1688] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:18 2022] DELETE /identity/v3/projects/0993d5226b264f3fa129a55e74f0eb5a => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:18.647541 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08eede-20be-4b91-b9be-8020528e8dcd admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:18.649145 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-fc08eede-20be-4b91-b9be-8020528e8dcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:18.663987 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aafe83-b553-42c8-97c3-4900b605cccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:18.664586 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aafe83-b553-42c8-97c3-4900b605cccd admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:18.664873 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aafe83-b553-42c8-97c3-4900b605cccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:18.665742 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aafe83-b553-42c8-97c3-4900b605cccd admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:18.666454 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aafe83-b553-42c8-97c3-4900b605cccd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:18.668082 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aafe83-b553-42c8-97c3-4900b605cccd admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:18.678572 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 829/1689] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:18.682249 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aafe83-b553-42c8-97c3-4900b605cccd admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:18.683933 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-46aafe83-b553-42c8-97c3-4900b605cccd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:18.696993 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aafe83-b553-42c8-97c3-4900b605cccd admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:18.701953 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bfa80-f4b6-4f33-8725-9f17ac80dab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:18.702630 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bfa80-f4b6-4f33-8725-9f17ac80dab2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:18.702920 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bfa80-f4b6-4f33-8725-9f17ac80dab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:18.703230 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bfa80-f4b6-4f33-8725-9f17ac80dab2 admin admin] PATH_INFO: `/v3/projects/26287d69715e40d5a8c666a976c2ac3e/users/bce1abf62a5a4dfeaf704e18823c56dd/roles/926b9f36c24847c582c84af904250abc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:18.725069 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08eede-20be-4b91-b9be-8020528e8dcd admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:18.726196 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6bfa80-f4b6-4f33-8725-9f17ac80dab2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26287d69715e40d5a8c666a976c2ac3e, user_id=bce1abf62a5a4dfeaf704e18823c56dd, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:18.730164 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6bfa80-f4b6-4f33-8725-9f17ac80dab2 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:18.731994 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e9d71-7d37-4c04-9c0f-a1b60316d59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:18.732769 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e9d71-7d37-4c04-9c0f-a1b60316d59d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:18.733060 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e9d71-7d37-4c04-9c0f-a1b60316d59d admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:18.733344 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e9d71-7d37-4c04-9c0f-a1b60316d59d admin admin] PATH_INFO: `/v3/users/47f31c68f0be45d3b415c3afcb2f3eb0` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:18.734554 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21e9d71-7d37-4c04-9c0f-a1b60316d59d admin admin] RBAC: Authorizing `identity:delete_user(user_id=47f31c68f0be45d3b415c3afcb2f3eb0)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:18.736633 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21e9d71-7d37-4c04-9c0f-a1b60316d59d admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:18.753638 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 830/1690] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:18 2022] PUT /identity/v3/projects/26287d69715e40d5a8c666a976c2ac3e/users/bce1abf62a5a4dfeaf704e18823c56dd/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:18.756784 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:18.765761 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75373144-b505-4395-8810-7787faf453c5 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:18.766060 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75373144-b505-4395-8810-7787faf453c5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:18.766375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75373144-b505-4395-8810-7787faf453c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:18.808055 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-75373144-b505-4395-8810-7787faf453c5 None None] MFA Rules not processed for user `bce1abf62a5a4dfeaf704e18823c56dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:18.809809 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-75373144-b505-4395-8810-7787faf453c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:18.835267 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-c21e9d71-7d37-4c04-9c0f-a1b60316d59d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47f31c68f0be45d3b415c3afcb2f3eb0'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:18.849262 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-c21e9d71-7d37-4c04-9c0f-a1b60316d59d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47f31c68f0be45d3b415c3afcb2f3eb0'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:18.875045 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-c21e9d71-7d37-4c04-9c0f-a1b60316d59d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47f31c68f0be45d3b415c3afcb2f3eb0'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:18.887660 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-c21e9d71-7d37-4c04-9c0f-a1b60316d59d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47f31c68f0be45d3b415c3afcb2f3eb0'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:18.888555 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-c21e9d71-7d37-4c04-9c0f-a1b60316d59d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47f31c68f0be45d3b415c3afcb2f3eb0'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:18.913507 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 831/1691] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:18.918231 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 861/1692] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:18 2022] DELETE /identity/v3/users/47f31c68f0be45d3b415c3afcb2f3eb0 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:18.939531 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75373144-b505-4395-8810-7787faf453c5 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:18.946071 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-75373144-b505-4395-8810-7787faf453c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:19.028062 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75373144-b505-4395-8810-7787faf453c5 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:19.037280 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:19.100275 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a07723c-809d-4cb1-9b48-66e4649578bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:19.101078 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a07723c-809d-4cb1-9b48-66e4649578bc service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:19.101360 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a07723c-809d-4cb1-9b48-66e4649578bc service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:19.101634 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a07723c-809d-4cb1-9b48-66e4649578bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:19.104062 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6a07723c-809d-4cb1-9b48-66e4649578bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:19.162965 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a07723c-809d-4cb1-9b48-66e4649578bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:19.164806 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a07723c-809d-4cb1-9b48-66e4649578bc service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:19.174813 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 832/1693] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:19.213333 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e9d71-7d37-4c04-9c0f-a1b60316d59d admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:19.215248 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-c21e9d71-7d37-4c04-9c0f-a1b60316d59d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:19.311045 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e9d71-7d37-4c04-9c0f-a1b60316d59d admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:19.315636 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20323492-137d-4859-93d0-3b47c216be23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:19.316288 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20323492-137d-4859-93d0-3b47c216be23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:19.316596 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20323492-137d-4859-93d0-3b47c216be23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:19.316883 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20323492-137d-4859-93d0-3b47c216be23 admin admin] PATH_INFO: `/v3/projects/6451650d419347f8b215b80ba3cb4bd6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:19.318052 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20323492-137d-4859-93d0-3b47c216be23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6451650d419347f8b215b80ba3cb4bd6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:19.319952 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20323492-137d-4859-93d0-3b47c216be23 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:19.369117 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-20323492-137d-4859-93d0-3b47c216be23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6451650d419347f8b215b80ba3cb4bd6', 'request_id': 'req-20323492-137d-4859-93d0-3b47c216be23'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:19.375303 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-20323492-137d-4859-93d0-3b47c216be23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6451650d419347f8b215b80ba3cb4bd6', 'request_id': 'req-20323492-137d-4859-93d0-3b47c216be23'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:19.380977 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-20323492-137d-4859-93d0-3b47c216be23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6451650d419347f8b215b80ba3cb4bd6', 'request_id': 'req-20323492-137d-4859-93d0-3b47c216be23'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:19.403197 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a07723c-809d-4cb1-9b48-66e4649578bc service nova] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:19.404413 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-20323492-137d-4859-93d0-3b47c216be23 admin admin] The token cache is being invalidate because project 6451650d419347f8b215b80ba3cb4bd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:19.404719 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6a07723c-809d-4cb1-9b48-66e4649578bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:19.404830 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-20323492-137d-4859-93d0-3b47c216be23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:19.405373 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-20323492-137d-4859-93d0-3b47c216be23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:19.406909 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 862/1694] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:19 2022] DELETE /identity/v3/projects/6451650d419347f8b215b80ba3cb4bd6 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:19.411421 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c068c31-fa40-4da1-bb7d-6f37fa93f765 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:19.411771 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c068c31-fa40-4da1-bb7d-6f37fa93f765 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:19.412119 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c068c31-fa40-4da1-bb7d-6f37fa93f765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:19.422246 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a07723c-809d-4cb1-9b48-66e4649578bc service nova] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:19.437174 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-7c068c31-fa40-4da1-bb7d-6f37fa93f765 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:19.439315 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-7c068c31-fa40-4da1-bb7d-6f37fa93f765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:19.447965 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:19.487660 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6846fe-0ac0-49b1-9911-994217045e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:19.488497 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6846fe-0ac0-49b1-9911-994217045e62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:19.488812 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6846fe-0ac0-49b1-9911-994217045e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:19.489090 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6846fe-0ac0-49b1-9911-994217045e62 admin admin] PATH_INFO: `/v3/users/bce1abf62a5a4dfeaf704e18823c56dd` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:19.490275 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6846fe-0ac0-49b1-9911-994217045e62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bce1abf62a5a4dfeaf704e18823c56dd)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:19.492235 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6846fe-0ac0-49b1-9911-994217045e62 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:19.513624 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 863/1695] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:19.521195 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780b8c8-4e90-44c9-9c53-b68f66462a27 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:19.521488 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780b8c8-4e90-44c9-9c53-b68f66462a27 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:19.521767 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780b8c8-4e90-44c9-9c53-b68f66462a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:19.542212 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-6780b8c8-4e90-44c9-9c53-b68f66462a27 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:19.543586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-6780b8c8-4e90-44c9-9c53-b68f66462a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:19.556800 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 864/1696] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:19.557665 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780b8c8-4e90-44c9-9c53-b68f66462a27 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:19.561560 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:19.563174 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ab6846fe-0ac0-49b1-9911-994217045e62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bce1abf62a5a4dfeaf704e18823c56dd'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:19.567311 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780b8c8-4e90-44c9-9c53-b68f66462a27 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:19.573047 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44273990-994b-45ab-a229-542048e2c5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:19.573678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44273990-994b-45ab-a229-542048e2c5f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:19.573939 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44273990-994b-45ab-a229-542048e2c5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:19.574212 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44273990-994b-45ab-a229-542048e2c5f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:19.574664 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ab6846fe-0ac0-49b1-9911-994217045e62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bce1abf62a5a4dfeaf704e18823c56dd'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:19.574909 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44273990-994b-45ab-a229-542048e2c5f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:19.575214 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44273990-994b-45ab-a229-542048e2c5f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:19.575392 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ab6846fe-0ac0-49b1-9911-994217045e62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bce1abf62a5a4dfeaf704e18823c56dd'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:19.576836 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44273990-994b-45ab-a229-542048e2c5f0 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:19.584927 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ab6846fe-0ac0-49b1-9911-994217045e62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bce1abf62a5a4dfeaf704e18823c56dd'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:19.585566 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ab6846fe-0ac0-49b1-9911-994217045e62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bce1abf62a5a4dfeaf704e18823c56dd'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:19.589774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 865/1697] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:09:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:19.594206 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44273990-994b-45ab-a229-542048e2c5f0 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:19.596403 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-44273990-994b-45ab-a229-542048e2c5f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:19.611471 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 833/1698] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:19 2022] DELETE /identity/v3/users/bce1abf62a5a4dfeaf704e18823c56dd => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:19.613007 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44273990-994b-45ab-a229-542048e2c5f0 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:19.617823 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b349ea-aac9-4079-b70b-29b249aba8f8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:19.618979 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b349ea-aac9-4079-b70b-29b249aba8f8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:19.619606 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b349ea-aac9-4079-b70b-29b249aba8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:19.641931 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-85b349ea-aac9-4079-b70b-29b249aba8f8 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:19.643385 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-85b349ea-aac9-4079-b70b-29b249aba8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:19.710703 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2752d920-6f37-459c-b4c8-eedcd4c92a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:19.712380 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752d920-6f37-459c-b4c8-eedcd4c92a12 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:19.712722 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752d920-6f37-459c-b4c8-eedcd4c92a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:19.713002 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752d920-6f37-459c-b4c8-eedcd4c92a12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:19.713787 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2752d920-6f37-459c-b4c8-eedcd4c92a12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:19.717201 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2752d920-6f37-459c-b4c8-eedcd4c92a12 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:19.755489 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 866/1699] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:19 2022] POST /identity/v3/projects => generated 363 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:09:19.756579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2752d920-6f37-459c-b4c8-eedcd4c92a12 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:19.761651 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 834/1700] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:19.774920 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b349ea-aac9-4079-b70b-29b249aba8f8 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:19.774920 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-85b349ea-aac9-4079-b70b-29b249aba8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:19.832884 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2752d920-6f37-459c-b4c8-eedcd4c92a12 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:19.838385 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee4f8f-4d76-4485-afbf-de4fb2346931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:19.839140 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee4f8f-4d76-4485-afbf-de4fb2346931 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:19.839426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee4f8f-4d76-4485-afbf-de4fb2346931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:19.839709 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee4f8f-4d76-4485-afbf-de4fb2346931 admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:19.840462 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ee4f8f-4d76-4485-afbf-de4fb2346931 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:19.842817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ee4f8f-4d76-4485-afbf-de4fb2346931 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:19.847364 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b349ea-aac9-4079-b70b-29b249aba8f8 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:19.892658 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 867/1701] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:19 2022] POST /identity/v3/users => generated 282 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:19.897409 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee4f8f-4d76-4485-afbf-de4fb2346931 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:19.918005 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee4f8f-4d76-4485-afbf-de4fb2346931 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:19.923804 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56641c39-051b-4481-9aa5-3102a63f6b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:19.924663 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56641c39-051b-4481-9aa5-3102a63f6b5c admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:19.925112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56641c39-051b-4481-9aa5-3102a63f6b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:19.925558 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56641c39-051b-4481-9aa5-3102a63f6b5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:19.926437 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56641c39-051b-4481-9aa5-3102a63f6b5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:19.928237 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56641c39-051b-4481-9aa5-3102a63f6b5c admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:19.931527 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:19.940978 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef1550-bccc-4e00-8093-7ce71cf5aabf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:19.942032 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef1550-bccc-4e00-8093-7ce71cf5aabf service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:19.942550 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef1550-bccc-4e00-8093-7ce71cf5aabf service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:19.943181 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef1550-bccc-4e00-8093-7ce71cf5aabf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:19.944839 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 868/1702] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:19.945930 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56641c39-051b-4481-9aa5-3102a63f6b5c admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:19.969392 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56641c39-051b-4481-9aa5-3102a63f6b5c admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:19.977939 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec66685e-b57c-4cd1-9416-200eaee92cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:19.978576 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec66685e-b57c-4cd1-9416-200eaee92cd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:19.978860 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec66685e-b57c-4cd1-9416-200eaee92cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:19.979137 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec66685e-b57c-4cd1-9416-200eaee92cd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:19.980801 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec66685e-b57c-4cd1-9416-200eaee92cd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:19.982590 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec66685e-b57c-4cd1-9416-200eaee92cd2 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:19.988628 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ef1550-bccc-4e00-8093-7ce71cf5aabf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:19.991219 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ef1550-bccc-4e00-8093-7ce71cf5aabf service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:20.001391 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 869/1703] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:20.002321 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec66685e-b57c-4cd1-9416-200eaee92cd2 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:20.016355 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec66685e-b57c-4cd1-9416-200eaee92cd2 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:20.023571 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 835/1704] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:09:20.025269 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b7741-df08-4eaa-ae5a-8ecb543f6463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:20.026132 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b7741-df08-4eaa-ae5a-8ecb543f6463 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:20.026566 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b7741-df08-4eaa-ae5a-8ecb543f6463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:20.026851 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b7741-df08-4eaa-ae5a-8ecb543f6463 admin admin] PATH_INFO: `/v3/projects/e812742efc4b4c1f9474622f10d59190/users/ae64858d732d48159d5989684f2ad818/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:20.046310 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8b7741-df08-4eaa-ae5a-8ecb543f6463 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e812742efc4b4c1f9474622f10d59190, user_id=ae64858d732d48159d5989684f2ad818, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:20.049778 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8b7741-df08-4eaa-ae5a-8ecb543f6463 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:20.080941 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 870/1705] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:20 2022] PUT /identity/v3/projects/e812742efc4b4c1f9474622f10d59190/users/ae64858d732d48159d5989684f2ad818/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:20.094148 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f56d968-1145-44cd-b1e7-9173a24978e3 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:20.094449 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f56d968-1145-44cd-b1e7-9173a24978e3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:20.094970 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f56d968-1145-44cd-b1e7-9173a24978e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:20.142967 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-9f56d968-1145-44cd-b1e7-9173a24978e3 None None] MFA Rules not processed for user `ae64858d732d48159d5989684f2ad818`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:20.144382 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9f56d968-1145-44cd-b1e7-9173a24978e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:20.170268 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:20.262911 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 836/1706] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:20 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:20.267847 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b7741-df08-4eaa-ae5a-8ecb543f6463 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:20.346264 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b7741-df08-4eaa-ae5a-8ecb543f6463 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:20.351839 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2ef1d-100d-4df5-8f8c-92ffa6386680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:20.353067 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2ef1d-100d-4df5-8f8c-92ffa6386680 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:20.353354 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2ef1d-100d-4df5-8f8c-92ffa6386680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:20.353636 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2ef1d-100d-4df5-8f8c-92ffa6386680 admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:20.355153 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c2ef1d-100d-4df5-8f8c-92ffa6386680 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:20.357095 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c2ef1d-100d-4df5-8f8c-92ffa6386680 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:20.385808 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 871/1707] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:20 2022] POST /identity/v3/projects => generated 363 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:20.390400 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f56d968-1145-44cd-b1e7-9173a24978e3 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:20.402895 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f56d968-1145-44cd-b1e7-9173a24978e3 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:20.433086 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2ef1d-100d-4df5-8f8c-92ffa6386680 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:20.434922 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-62c2ef1d-100d-4df5-8f8c-92ffa6386680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:20.471376 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1f3d6-6a50-4462-88de-af5fa8a98eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:20.472016 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1f3d6-6a50-4462-88de-af5fa8a98eb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:20.472308 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1f3d6-6a50-4462-88de-af5fa8a98eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:20.472624 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1f3d6-6a50-4462-88de-af5fa8a98eb4 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:20.473326 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f1f3d6-6a50-4462-88de-af5fa8a98eb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:20.475294 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f1f3d6-6a50-4462-88de-af5fa8a98eb4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:20.509080 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 837/1708] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:20 2022] POST /identity/v3/users => generated 288 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:20.510634 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1f3d6-6a50-4462-88de-af5fa8a98eb4 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:20.525411 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1f3d6-6a50-4462-88de-af5fa8a98eb4 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:20.533337 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0887b450-a5af-4daa-b04b-5463cde8d065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:20.536089 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2ef1d-100d-4df5-8f8c-92ffa6386680 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:20.536363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0887b450-a5af-4daa-b04b-5463cde8d065 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:20.536798 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0887b450-a5af-4daa-b04b-5463cde8d065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:20.537226 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0887b450-a5af-4daa-b04b-5463cde8d065 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:20.538747 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0887b450-a5af-4daa-b04b-5463cde8d065 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:20.541549 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0887b450-a5af-4daa-b04b-5463cde8d065 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:20.543281 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055532e-bad2-421e-aa1f-f5fe5446cb0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:20.545008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055532e-bad2-421e-aa1f-f5fe5446cb0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:20.545455 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055532e-bad2-421e-aa1f-f5fe5446cb0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:20.545957 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055532e-bad2-421e-aa1f-f5fe5446cb0e admin admin] PATH_INFO: `/v3/projects/26287d69715e40d5a8c666a976c2ac3e` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:20.547367 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c055532e-bad2-421e-aa1f-f5fe5446cb0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=26287d69715e40d5a8c666a976c2ac3e)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:20.550254 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c055532e-bad2-421e-aa1f-f5fe5446cb0e admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:20.556520 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 838/1709] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:20.558476 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0887b450-a5af-4daa-b04b-5463cde8d065 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:20.595873 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0887b450-a5af-4daa-b04b-5463cde8d065 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:20.604236 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12795740-7c99-4cff-8518-7d83960659c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:20.605381 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12795740-7c99-4cff-8518-7d83960659c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:20.606448 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12795740-7c99-4cff-8518-7d83960659c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:20.606837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12795740-7c99-4cff-8518-7d83960659c3 admin admin] PATH_INFO: `/v3/projects/7a0c5317455e4f4a8bf6bcd53c957638/users/4f11253ea4044bcaba0fe2b58215e402/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:20.614546 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-c055532e-bad2-421e-aa1f-f5fe5446cb0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26287d69715e40d5a8c666a976c2ac3e', 'request_id': 'req-c055532e-bad2-421e-aa1f-f5fe5446cb0e'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:20.615785 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:20.628915 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-c055532e-bad2-421e-aa1f-f5fe5446cb0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26287d69715e40d5a8c666a976c2ac3e', 'request_id': 'req-c055532e-bad2-421e-aa1f-f5fe5446cb0e'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:20.642414 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-c055532e-bad2-421e-aa1f-f5fe5446cb0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26287d69715e40d5a8c666a976c2ac3e', 'request_id': 'req-c055532e-bad2-421e-aa1f-f5fe5446cb0e'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:20.674304 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-c055532e-bad2-421e-aa1f-f5fe5446cb0e admin admin] The token cache is being invalidate because project 26287d69715e40d5a8c666a976c2ac3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:20.681723 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-c055532e-bad2-421e-aa1f-f5fe5446cb0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:20.682261 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12795740-7c99-4cff-8518-7d83960659c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a0c5317455e4f4a8bf6bcd53c957638, user_id=4f11253ea4044bcaba0fe2b58215e402, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:20.684415 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-c055532e-bad2-421e-aa1f-f5fe5446cb0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:20.686246 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12795740-7c99-4cff-8518-7d83960659c3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:20.689417 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 872/1710] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:20 2022] DELETE /identity/v3/projects/26287d69715e40d5a8c666a976c2ac3e => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:20.697026 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c1a93-53de-404e-9821-bb9c4fcee508 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:20.699020 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c1a93-53de-404e-9821-bb9c4fcee508 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:20.700991 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c1a93-53de-404e-9821-bb9c4fcee508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:20.740408 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-047c1a93-53de-404e-9821-bb9c4fcee508 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:20.745356 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-047c1a93-53de-404e-9821-bb9c4fcee508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:20.751446 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 839/1711] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:20 2022] PUT /identity/v3/projects/7a0c5317455e4f4a8bf6bcd53c957638/users/4f11253ea4044bcaba0fe2b58215e402/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:20.755996 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12795740-7c99-4cff-8518-7d83960659c3 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:20.757759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-12795740-7c99-4cff-8518-7d83960659c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:20.889725 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 873/1712] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:20.899086 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12795740-7c99-4cff-8518-7d83960659c3 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:20.901805 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82c7ea-6257-4ece-80f0-556fb02135fb None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:20.902602 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82c7ea-6257-4ece-80f0-556fb02135fb None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:20.902916 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82c7ea-6257-4ece-80f0-556fb02135fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:20.917746 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d673b7e8-86d9-45a4-965e-3476d387293e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:20.918633 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d673b7e8-86d9-45a4-965e-3476d387293e admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:20.918935 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d673b7e8-86d9-45a4-965e-3476d387293e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:20.919218 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d673b7e8-86d9-45a4-965e-3476d387293e admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:20.919935 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d673b7e8-86d9-45a4-965e-3476d387293e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:20.925160 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d673b7e8-86d9-45a4-965e-3476d387293e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:20.930014 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-dd82c7ea-6257-4ece-80f0-556fb02135fb None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:20.931372 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-dd82c7ea-6257-4ece-80f0-556fb02135fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:20.948242 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 840/1713] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:20.953100 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 874/1714] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:20.954140 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd82c7ea-6257-4ece-80f0-556fb02135fb None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:20.959838 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d673b7e8-86d9-45a4-965e-3476d387293e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:20.969588 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:20.973535 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd82c7ea-6257-4ece-80f0-556fb02135fb None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:20.980828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d673b7e8-86d9-45a4-965e-3476d387293e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:20.981155 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334396f-8e8b-494d-99b3-6ef379b854fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:20.981859 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334396f-8e8b-494d-99b3-6ef379b854fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:20.982156 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334396f-8e8b-494d-99b3-6ef379b854fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:20.982449 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334396f-8e8b-494d-99b3-6ef379b854fe admin admin] PATH_INFO: `/v3/projects/7a0c5317455e4f4a8bf6bcd53c957638/users/4f11253ea4044bcaba0fe2b58215e402/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:20.984680 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7334396f-8e8b-494d-99b3-6ef379b854fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a0c5317455e4f4a8bf6bcd53c957638, user_id=4f11253ea4044bcaba0fe2b58215e402, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:20.987365 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7334396f-8e8b-494d-99b3-6ef379b854fe admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:20.988775 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c412f23b-fcbc-4c1a-ae05-cd67207fbec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:20.989639 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412f23b-fcbc-4c1a-ae05-cd67207fbec4 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:20.990350 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412f23b-fcbc-4c1a-ae05-cd67207fbec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:20.990906 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412f23b-fcbc-4c1a-ae05-cd67207fbec4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:20.991888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c412f23b-fcbc-4c1a-ae05-cd67207fbec4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:20.992395 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c412f23b-fcbc-4c1a-ae05-cd67207fbec4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:20.994354 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c412f23b-fcbc-4c1a-ae05-cd67207fbec4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:21.015847 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 875/1715] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:20 2022] PUT /identity/v3/projects/7a0c5317455e4f4a8bf6bcd53c957638/users/4f11253ea4044bcaba0fe2b58215e402/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:21.016225 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 841/1716] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:09:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:09:21.028454 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334396f-8e8b-494d-99b3-6ef379b854fe admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:21.030114 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d7e75-1504-4cab-96a1-378e0e28d674 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:21.030479 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d7e75-1504-4cab-96a1-378e0e28d674 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:21.030759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d7e75-1504-4cab-96a1-378e0e28d674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:21.076728 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-2f3d7e75-1504-4cab-96a1-378e0e28d674 None None] MFA Rules not processed for user `4f11253ea4044bcaba0fe2b58215e402`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:21.078603 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2f3d7e75-1504-4cab-96a1-378e0e28d674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:21.147958 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334396f-8e8b-494d-99b3-6ef379b854fe admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:21.160163 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf680e-d693-48d2-b2fa-7a451ef4f28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:21.161029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf680e-d693-48d2-b2fa-7a451ef4f28f admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:21.161364 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf680e-d693-48d2-b2fa-7a451ef4f28f admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:21.161790 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf680e-d693-48d2-b2fa-7a451ef4f28f admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:21.162447 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbf680e-d693-48d2-b2fa-7a451ef4f28f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:21.165751 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbf680e-d693-48d2-b2fa-7a451ef4f28f admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:21.213621 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 876/1717] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:21 2022] POST /identity/v3/projects => generated 385 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:21.217854 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf680e-d693-48d2-b2fa-7a451ef4f28f admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:21.224504 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 842/1718] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:21 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:21.246058 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d7e75-1504-4cab-96a1-378e0e28d674 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:21.253984 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2f3d7e75-1504-4cab-96a1-378e0e28d674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:21.326748 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:21.353024 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d7e75-1504-4cab-96a1-378e0e28d674 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:21.362488 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf680e-d693-48d2-b2fa-7a451ef4f28f admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:21.369915 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc002e-adc0-43d3-b6e2-41b9390f331a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:21.371511 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc002e-adc0-43d3-b6e2-41b9390f331a admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:21.372125 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc002e-adc0-43d3-b6e2-41b9390f331a admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:21.372125 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc002e-adc0-43d3-b6e2-41b9390f331a admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:21.373460 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddc002e-adc0-43d3-b6e2-41b9390f331a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:21.376408 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddc002e-adc0-43d3-b6e2-41b9390f331a admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:21.413310 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 877/1719] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:21 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:21.415770 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc002e-adc0-43d3-b6e2-41b9390f331a admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:21.427737 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc002e-adc0-43d3-b6e2-41b9390f331a admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:21.438613 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c74bad-b36b-45ba-b4be-2dacdd4dd4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:21.439446 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c74bad-b36b-45ba-b4be-2dacdd4dd4fd admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:21.439737 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c74bad-b36b-45ba-b4be-2dacdd4dd4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:21.440018 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c74bad-b36b-45ba-b4be-2dacdd4dd4fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:21.440758 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c74bad-b36b-45ba-b4be-2dacdd4dd4fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:21.442363 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c74bad-b36b-45ba-b4be-2dacdd4dd4fd admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:21.461377 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2bc137-5696-4574-9e82-ba45e7fd227e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:21.463003 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2bc137-5696-4574-9e82-ba45e7fd227e service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:21.463459 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2bc137-5696-4574-9e82-ba45e7fd227e service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:21.463692 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2bc137-5696-4574-9e82-ba45e7fd227e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:21.464167 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 878/1720] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:21.465446 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c74bad-b36b-45ba-b4be-2dacdd4dd4fd admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:21.485512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c74bad-b36b-45ba-b4be-2dacdd4dd4fd admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:21.492210 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19b5f2-f276-43fc-83fe-ba761bf0e96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:21.493615 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19b5f2-f276-43fc-83fe-ba761bf0e96e admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:21.493971 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19b5f2-f276-43fc-83fe-ba761bf0e96e admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:21.494252 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19b5f2-f276-43fc-83fe-ba761bf0e96e admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:21.495445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab19b5f2-f276-43fc-83fe-ba761bf0e96e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:21.500687 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab19b5f2-f276-43fc-83fe-ba761bf0e96e admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:21.525258 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 879/1721] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:21.539057 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19b5f2-f276-43fc-83fe-ba761bf0e96e admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:21.571145 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19b5f2-f276-43fc-83fe-ba761bf0e96e admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:21.583909 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c6464-63bc-4744-8b05-1f64e147f074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:21.585481 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:21.587744 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c6464-63bc-4744-8b05-1f64e147f074 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:21.588059 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c6464-63bc-4744-8b05-1f64e147f074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:21.588346 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c6464-63bc-4744-8b05-1f64e147f074 admin admin] PATH_INFO: `/v3/projects/2cd41b6c817b4d70819489684de572cd/users/ad58dcf861c8475ebaae8caf0883d7b9/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:21.605754 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2bc137-5696-4574-9e82-ba45e7fd227e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:21.607488 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2bc137-5696-4574-9e82-ba45e7fd227e service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:21.632059 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 843/1722] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:21 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:21.639888 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c6464-63bc-4744-8b05-1f64e147f074 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cd41b6c817b4d70819489684de572cd, user_id=ad58dcf861c8475ebaae8caf0883d7b9, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:21.646787 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c6464-63bc-4744-8b05-1f64e147f074 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:21.679382 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 880/1723] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:21 2022] PUT /identity/v3/projects/2cd41b6c817b4d70819489684de572cd/users/ad58dcf861c8475ebaae8caf0883d7b9/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:21.690631 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8a7af-4864-46d0-98f6-3961ac7e0793 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:21.692360 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8a7af-4864-46d0-98f6-3961ac7e0793 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:21.692876 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8a7af-4864-46d0-98f6-3961ac7e0793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:21.799336 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-21e8a7af-4864-46d0-98f6-3961ac7e0793 None None] MFA Rules not processed for user `ad58dcf861c8475ebaae8caf0883d7b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:21.800697 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-21e8a7af-4864-46d0-98f6-3961ac7e0793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:22.071096 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 844/1724] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:22.082548 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c6464-63bc-4744-8b05-1f64e147f074 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:22.247368 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c6464-63bc-4744-8b05-1f64e147f074 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:22.252988 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c993a-e5ad-4c9c-92f4-4278b8c2f406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:22.253896 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c993a-e5ad-4c9c-92f4-4278b8c2f406 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:22.254392 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c993a-e5ad-4c9c-92f4-4278b8c2f406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:22.254812 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c993a-e5ad-4c9c-92f4-4278b8c2f406 admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:22.255654 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702c993a-e5ad-4c9c-92f4-4278b8c2f406 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:22.258093 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702c993a-e5ad-4c9c-92f4-4278b8c2f406 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:22.325691 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 881/1725] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:22 2022] POST /identity/v3/projects => generated 385 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:22.330793 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8a7af-4864-46d0-98f6-3961ac7e0793 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:22.350464 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8a7af-4864-46d0-98f6-3961ac7e0793 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:22.505648 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2a3867-e99e-4efa-8c53-73853876d425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:22.506320 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a3867-e99e-4efa-8c53-73853876d425 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:22.509268 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a3867-e99e-4efa-8c53-73853876d425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:22.510791 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a3867-e99e-4efa-8c53-73853876d425 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:22.512831 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2a3867-e99e-4efa-8c53-73853876d425 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:22.520103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2a3867-e99e-4efa-8c53-73853876d425 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:22.575047 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c993a-e5ad-4c9c-92f4-4278b8c2f406 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:22.575401 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 845/1726] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:22 2022] POST /identity/v3/users => generated 299 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:22.618961 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c993a-e5ad-4c9c-92f4-4278b8c2f406 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:22.647319 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794773e-5dc4-40db-b5d6-0862e7546feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:22.652976 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794773e-5dc4-40db-b5d6-0862e7546feb admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:22.653730 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794773e-5dc4-40db-b5d6-0862e7546feb admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:22.655258 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794773e-5dc4-40db-b5d6-0862e7546feb admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:22.658417 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c794773e-5dc4-40db-b5d6-0862e7546feb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:22.663617 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c794773e-5dc4-40db-b5d6-0862e7546feb admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:22.703523 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2a3867-e99e-4efa-8c53-73853876d425 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:22.716695 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 882/1727] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:22.723449 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794773e-5dc4-40db-b5d6-0862e7546feb admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:22.744057 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2a3867-e99e-4efa-8c53-73853876d425 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:22.762592 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764cc79a-79d8-40c4-b606-1926b52cb717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:22.765180 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764cc79a-79d8-40c4-b606-1926b52cb717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:22.765547 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794773e-5dc4-40db-b5d6-0862e7546feb admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:22.766803 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764cc79a-79d8-40c4-b606-1926b52cb717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:22.767123 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764cc79a-79d8-40c4-b606-1926b52cb717 admin admin] PATH_INFO: `/v3/users/ae64858d732d48159d5989684f2ad818` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:22.781325 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764cc79a-79d8-40c4-b606-1926b52cb717 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae64858d732d48159d5989684f2ad818)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:22.782845 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764cc79a-79d8-40c4-b606-1926b52cb717 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:22.790790 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92444adc-57e4-4ba3-8209-c307dc819cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:22.791549 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92444adc-57e4-4ba3-8209-c307dc819cb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:22.791774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92444adc-57e4-4ba3-8209-c307dc819cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:22.792082 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92444adc-57e4-4ba3-8209-c307dc819cb1 admin admin] PATH_INFO: `/v3/projects/833b90f7b4b747c3b5253f74535062d5/users/0eb53f5d7e73428a8ee8e723042fa550/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:22.852142 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92444adc-57e4-4ba3-8209-c307dc819cb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=833b90f7b4b747c3b5253f74535062d5, user_id=0eb53f5d7e73428a8ee8e723042fa550, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:22.857841 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92444adc-57e4-4ba3-8209-c307dc819cb1 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:22.912015 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 883/1728] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:22 2022] PUT /identity/v3/projects/833b90f7b4b747c3b5253f74535062d5/users/0eb53f5d7e73428a8ee8e723042fa550/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:22.916381 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92444adc-57e4-4ba3-8209-c307dc819cb1 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:22.921485 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-764cc79a-79d8-40c4-b606-1926b52cb717 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae64858d732d48159d5989684f2ad818'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:22.947056 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:22.955716 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-764cc79a-79d8-40c4-b606-1926b52cb717 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae64858d732d48159d5989684f2ad818'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:22.957723 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-764cc79a-79d8-40c4-b606-1926b52cb717 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae64858d732d48159d5989684f2ad818'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:22.980320 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-764cc79a-79d8-40c4-b606-1926b52cb717 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae64858d732d48159d5989684f2ad818'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:22.982125 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-764cc79a-79d8-40c4-b606-1926b52cb717 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae64858d732d48159d5989684f2ad818'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:23.003161 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:23.062487 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 846/1729] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:22 2022] DELETE /identity/v3/users/ae64858d732d48159d5989684f2ad818 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:23.068581 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c10ed-1c9b-4175-92cf-6978886b2a43 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:23.069414 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c10ed-1c9b-4175-92cf-6978886b2a43 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:23.069654 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92444adc-57e4-4ba3-8209-c307dc819cb1 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:23.070085 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c10ed-1c9b-4175-92cf-6978886b2a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:23.125036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-945c10ed-1c9b-4175-92cf-6978886b2a43 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:23.131675 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-945c10ed-1c9b-4175-92cf-6978886b2a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:23.257042 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47d944-4d17-43d1-b21f-a78608f02348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:23.257692 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47d944-4d17-43d1-b21f-a78608f02348 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:23.257979 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47d944-4d17-43d1-b21f-a78608f02348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:23.258254 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47d944-4d17-43d1-b21f-a78608f02348 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:23.259941 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a47d944-4d17-43d1-b21f-a78608f02348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:23.261256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a47d944-4d17-43d1-b21f-a78608f02348 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:23.279985 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 847/1730] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:23.285802 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 884/1731] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:23.287028 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c10ed-1c9b-4175-92cf-6978886b2a43 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:23.291851 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-945c10ed-1c9b-4175-92cf-6978886b2a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:23.313976 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c10ed-1c9b-4175-92cf-6978886b2a43 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:23.314192 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47d944-4d17-43d1-b21f-a78608f02348 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:23.316152 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-2a47d944-4d17-43d1-b21f-a78608f02348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:23.321998 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824b4783-1f79-4719-80e7-2ddadf7b3c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:23.322909 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b4783-1f79-4719-80e7-2ddadf7b3c09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:23.323513 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b4783-1f79-4719-80e7-2ddadf7b3c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:23.323910 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b4783-1f79-4719-80e7-2ddadf7b3c09 admin admin] PATH_INFO: `/v3/projects/833b90f7b4b747c3b5253f74535062d5/users/0eb53f5d7e73428a8ee8e723042fa550/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:23.330274 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824b4783-1f79-4719-80e7-2ddadf7b3c09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=833b90f7b4b747c3b5253f74535062d5, user_id=0eb53f5d7e73428a8ee8e723042fa550, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:23.337747 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824b4783-1f79-4719-80e7-2ddadf7b3c09 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:23.339357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47d944-4d17-43d1-b21f-a78608f02348 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:23.354263 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d994d-ef4d-4182-8c26-4003145318ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:23.355348 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d994d-ef4d-4182-8c26-4003145318ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:23.355709 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d994d-ef4d-4182-8c26-4003145318ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:23.356193 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d994d-ef4d-4182-8c26-4003145318ec admin admin] PATH_INFO: `/v3/users/14ef80d493364a4387b6556a339cbbb8` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:23.361262 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d994d-ef4d-4182-8c26-4003145318ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=14ef80d493364a4387b6556a339cbbb8)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:23.370449 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d994d-ef4d-4182-8c26-4003145318ec admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:23.404703 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 848/1732] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:23 2022] PUT /identity/v3/projects/833b90f7b4b747c3b5253f74535062d5/users/0eb53f5d7e73428a8ee8e723042fa550/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:23.406243 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824b4783-1f79-4719-80e7-2ddadf7b3c09 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:23.417747 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-824b4783-1f79-4719-80e7-2ddadf7b3c09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:23.536185 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-748d994d-ef4d-4182-8c26-4003145318ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14ef80d493364a4387b6556a339cbbb8'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:23.538268 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-748d994d-ef4d-4182-8c26-4003145318ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14ef80d493364a4387b6556a339cbbb8'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:23.573339 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-748d994d-ef4d-4182-8c26-4003145318ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14ef80d493364a4387b6556a339cbbb8'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:23.581950 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824b4783-1f79-4719-80e7-2ddadf7b3c09 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:23.586774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-748d994d-ef4d-4182-8c26-4003145318ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14ef80d493364a4387b6556a339cbbb8'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:23.587730 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-748d994d-ef4d-4182-8c26-4003145318ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14ef80d493364a4387b6556a339cbbb8'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:23.596322 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37c8f14-bb50-487b-8d62-6c2be68dd9c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:23.597408 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c8f14-bb50-487b-8d62-6c2be68dd9c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:23.597893 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c8f14-bb50-487b-8d62-6c2be68dd9c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:23.598358 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c8f14-bb50-487b-8d62-6c2be68dd9c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:23.601333 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a37c8f14-bb50-487b-8d62-6c2be68dd9c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:23.613973 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 885/1733] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:23 2022] DELETE /identity/v3/users/14ef80d493364a4387b6556a339cbbb8 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:23.617083 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db753cec-3fa0-458e-8385-76a0402395c5 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:23.617460 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db753cec-3fa0-458e-8385-76a0402395c5 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:23.618380 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db753cec-3fa0-458e-8385-76a0402395c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:23.730232 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-db753cec-3fa0-458e-8385-76a0402395c5 None None] MFA Rules not processed for user `0eb53f5d7e73428a8ee8e723042fa550`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:23.734973 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-db753cec-3fa0-458e-8385-76a0402395c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:23.777197 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37c8f14-bb50-487b-8d62-6c2be68dd9c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:23.780963 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37c8f14-bb50-487b-8d62-6c2be68dd9c2 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:23.821915 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 849/1734] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:23.823516 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0bb07-6666-4f79-adef-4c664ca918be None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:23.827280 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0bb07-6666-4f79-adef-4c664ca918be None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:23.827701 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0bb07-6666-4f79-adef-4c664ca918be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:23.874547 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4dd0bb07-6666-4f79-adef-4c664ca918be None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:23.876170 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4dd0bb07-6666-4f79-adef-4c664ca918be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:23.881088 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:23.927803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:24.016460 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 886/1735] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:23 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 402 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:24.029945 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db753cec-3fa0-458e-8385-76a0402395c5 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:24.032942 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-db753cec-3fa0-458e-8385-76a0402395c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:24.039112 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 850/1736] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:24.077395 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b75453-8945-45fe-b02b-ca252eff34de None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:24.078270 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b75453-8945-45fe-b02b-ca252eff34de None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:24.078758 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b75453-8945-45fe-b02b-ca252eff34de None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:24.081075 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 851/1737] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 14:09:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:09:24.082249 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b75453-8945-45fe-b02b-ca252eff34de None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:24.084924 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-99b75453-8945-45fe-b02b-ca252eff34de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:24.147819 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db753cec-3fa0-458e-8385-76a0402395c5 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:24.165679 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081c8e3-1c64-4537-9f15-6bfd6235c4a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:24.166375 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081c8e3-1c64-4537-9f15-6bfd6235c4a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:24.166888 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081c8e3-1c64-4537-9f15-6bfd6235c4a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:24.167000 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081c8e3-1c64-4537-9f15-6bfd6235c4a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:24.185004 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d081c8e3-1c64-4537-9f15-6bfd6235c4a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:24.187739 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d081c8e3-1c64-4537-9f15-6bfd6235c4a0 service cinder] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:24.205734 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 887/1738] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:24 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:24.207179 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e147eb-1090-4f80-9307-542c1f7b49b0 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:24.207437 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e147eb-1090-4f80-9307-542c1f7b49b0 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:24.207722 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e147eb-1090-4f80-9307-542c1f7b49b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:24.234303 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-b6e147eb-1090-4f80-9307-542c1f7b49b0 None None] MFA Rules not processed for user `a246654c783b4cc68d212e91bb05d8c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:24.236758 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-b6e147eb-1090-4f80-9307-542c1f7b49b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:24.247409 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b75453-8945-45fe-b02b-ca252eff34de None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:24.252752 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f29c8-5f52-4a79-91f6-e74698ed7b08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:24.262168 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f29c8-5f52-4a79-91f6-e74698ed7b08 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:24.262168 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f29c8-5f52-4a79-91f6-e74698ed7b08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:24.262168 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f29c8-5f52-4a79-91f6-e74698ed7b08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:24.299133 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f29c8-5f52-4a79-91f6-e74698ed7b08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:24.301554 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f29c8-5f52-4a79-91f6-e74698ed7b08 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:24.316889 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 852/1739] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:24.318432 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f29c8-5f52-4a79-91f6-e74698ed7b08 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:24.322155 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9c2f29c8-5f52-4a79-91f6-e74698ed7b08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:24.356169 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 888/1740] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 14:09:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:24.359066 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f29c8-5f52-4a79-91f6-e74698ed7b08 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:24.368596 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d64dc20-fd95-4bfb-a88c-d238083cbc80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:24.369443 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64dc20-fd95-4bfb-a88c-d238083cbc80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:24.369828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64dc20-fd95-4bfb-a88c-d238083cbc80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:24.373663 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64dc20-fd95-4bfb-a88c-d238083cbc80 admin admin] PATH_INFO: `/v3/users/ad58dcf861c8475ebaae8caf0883d7b9` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:24.374980 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d64dc20-fd95-4bfb-a88c-d238083cbc80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad58dcf861c8475ebaae8caf0883d7b9)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:24.379814 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d64dc20-fd95-4bfb-a88c-d238083cbc80 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:24.453374 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6d64dc20-fd95-4bfb-a88c-d238083cbc80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad58dcf861c8475ebaae8caf0883d7b9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:24.464524 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6d64dc20-fd95-4bfb-a88c-d238083cbc80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad58dcf861c8475ebaae8caf0883d7b9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:24.466065 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6d64dc20-fd95-4bfb-a88c-d238083cbc80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad58dcf861c8475ebaae8caf0883d7b9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:24.493441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e147eb-1090-4f80-9307-542c1f7b49b0 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:24.494958 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-b6e147eb-1090-4f80-9307-542c1f7b49b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:24.498352 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6d64dc20-fd95-4bfb-a88c-d238083cbc80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad58dcf861c8475ebaae8caf0883d7b9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:24.499107 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6d64dc20-fd95-4bfb-a88c-d238083cbc80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad58dcf861c8475ebaae8caf0883d7b9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:24.520670 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e147eb-1090-4f80-9307-542c1f7b49b0 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:24.521524 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 853/1741] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:24 2022] DELETE /identity/v3/users/ad58dcf861c8475ebaae8caf0883d7b9 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:24.527433 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699fd613-ad35-4c5b-ab86-c28677beff04 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:24.528083 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699fd613-ad35-4c5b-ab86-c28677beff04 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:24.528494 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699fd613-ad35-4c5b-ab86-c28677beff04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:24.566332 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-699fd613-ad35-4c5b-ab86-c28677beff04 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:24.571346 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-699fd613-ad35-4c5b-ab86-c28677beff04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:24.601630 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:24.637713 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8b9a3-4e6a-40bc-a389-2c8e3a3f0e18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:24.638878 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8b9a3-4e6a-40bc-a389-2c8e3a3f0e18 service nova] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:24.638878 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8b9a3-4e6a-40bc-a389-2c8e3a3f0e18 service nova] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:24.639258 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8b9a3-4e6a-40bc-a389-2c8e3a3f0e18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:24.641802 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-2da8b9a3-4e6a-40bc-a389-2c8e3a3f0e18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:24.655870 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da8b9a3-4e6a-40bc-a389-2c8e3a3f0e18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:24.657656 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da8b9a3-4e6a-40bc-a389-2c8e3a3f0e18 service nova] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:24.663813 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:24.668512 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 854/1742] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:24.672896 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 889/1743] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:24.673416 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699fd613-ad35-4c5b-ab86-c28677beff04 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:24.674951 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-699fd613-ad35-4c5b-ab86-c28677beff04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:24.688339 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699fd613-ad35-4c5b-ab86-c28677beff04 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:24.694553 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc4694e-2f59-4360-bebb-15aebace39c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:24.695484 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4694e-2f59-4360-bebb-15aebace39c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:24.696697 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4694e-2f59-4360-bebb-15aebace39c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:24.697312 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4694e-2f59-4360-bebb-15aebace39c6 admin admin] PATH_INFO: `/v3/projects/e812742efc4b4c1f9474622f10d59190` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:24.698699 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc4694e-2f59-4360-bebb-15aebace39c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e812742efc4b4c1f9474622f10d59190)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:24.699557 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8b9a3-4e6a-40bc-a389-2c8e3a3f0e18 service nova] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:24.701172 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc4694e-2f59-4360-bebb-15aebace39c6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:24.704085 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-2da8b9a3-4e6a-40bc-a389-2c8e3a3f0e18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:24.808715 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-afc4694e-2f59-4360-bebb-15aebace39c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e812742efc4b4c1f9474622f10d59190', 'request_id': 'req-afc4694e-2f59-4360-bebb-15aebace39c6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:24.813243 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8b9a3-4e6a-40bc-a389-2c8e3a3f0e18 service nova] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:24.820844 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-afc4694e-2f59-4360-bebb-15aebace39c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e812742efc4b4c1f9474622f10d59190', 'request_id': 'req-afc4694e-2f59-4360-bebb-15aebace39c6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:24.827640 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-afc4694e-2f59-4360-bebb-15aebace39c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e812742efc4b4c1f9474622f10d59190', 'request_id': 'req-afc4694e-2f59-4360-bebb-15aebace39c6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:24.848081 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-afc4694e-2f59-4360-bebb-15aebace39c6 admin admin] The token cache is being invalidate because project e812742efc4b4c1f9474622f10d59190 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:24.848930 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-afc4694e-2f59-4360-bebb-15aebace39c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:24.851506 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-afc4694e-2f59-4360-bebb-15aebace39c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:24.856565 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 855/1744] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:24 2022] DELETE /identity/v3/projects/e812742efc4b4c1f9474622f10d59190 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:24.860741 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc4694e-2f59-4360-bebb-15aebace39c6 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:24.862743 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-afc4694e-2f59-4360-bebb-15aebace39c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:24.897085 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ece39-82a9-4859-be0f-587271a001f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:24.897924 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ece39-82a9-4859-be0f-587271a001f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:24.898375 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ece39-82a9-4859-be0f-587271a001f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:24.899307 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ece39-82a9-4859-be0f-587271a001f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:24.901577 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-fd0ece39-82a9-4859-be0f-587271a001f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:24.943586 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc4694e-2f59-4360-bebb-15aebace39c6 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:24.949341 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a50b4-df5a-4089-9907-0963fc876cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:24.949998 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a50b4-df5a-4089-9907-0963fc876cde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:24.950344 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a50b4-df5a-4089-9907-0963fc876cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:24.951096 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a50b4-df5a-4089-9907-0963fc876cde admin admin] PATH_INFO: `/v3/users/4f11253ea4044bcaba0fe2b58215e402` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:24.952357 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1a50b4-df5a-4089-9907-0963fc876cde admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f11253ea4044bcaba0fe2b58215e402)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:24.954313 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1a50b4-df5a-4089-9907-0963fc876cde admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:25.017731 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ece39-82a9-4859-be0f-587271a001f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:25.020453 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ece39-82a9-4859-be0f-587271a001f0 service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:25.028490 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0f1a50b4-df5a-4089-9907-0963fc876cde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f11253ea4044bcaba0fe2b58215e402'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:25.041422 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:25.051574 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 890/1745] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:25.060801 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0f1a50b4-df5a-4089-9907-0963fc876cde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f11253ea4044bcaba0fe2b58215e402'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:25.062074 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0f1a50b4-df5a-4089-9907-0963fc876cde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f11253ea4044bcaba0fe2b58215e402'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:25.075805 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0f1a50b4-df5a-4089-9907-0963fc876cde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f11253ea4044bcaba0fe2b58215e402'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:25.076691 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0f1a50b4-df5a-4089-9907-0963fc876cde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f11253ea4044bcaba0fe2b58215e402'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:25.099111 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 856/1746] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:24 2022] DELETE /identity/v3/users/4f11253ea4044bcaba0fe2b58215e402 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:25.224610 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ece39-82a9-4859-be0f-587271a001f0 service neutron] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:25.226309 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-fd0ece39-82a9-4859-be0f-587271a001f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:25.254384 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ece39-82a9-4859-be0f-587271a001f0 service neutron] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:25.354404 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42638fb-b412-4f23-995b-61e596a34ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:25.355290 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42638fb-b412-4f23-995b-61e596a34ad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:25.355675 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42638fb-b412-4f23-995b-61e596a34ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:25.356050 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42638fb-b412-4f23-995b-61e596a34ad8 admin admin] PATH_INFO: `/v3/projects/97ba78c7fc184b549caa84cf6a4f0df1` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:25.357622 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42638fb-b412-4f23-995b-61e596a34ad8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97ba78c7fc184b549caa84cf6a4f0df1)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:25.360781 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42638fb-b412-4f23-995b-61e596a34ad8 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:25.374845 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:25.444824 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-e42638fb-b412-4f23-995b-61e596a34ad8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97ba78c7fc184b549caa84cf6a4f0df1', 'request_id': 'req-e42638fb-b412-4f23-995b-61e596a34ad8'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:25.452974 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-e42638fb-b412-4f23-995b-61e596a34ad8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97ba78c7fc184b549caa84cf6a4f0df1', 'request_id': 'req-e42638fb-b412-4f23-995b-61e596a34ad8'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:25.459329 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-e42638fb-b412-4f23-995b-61e596a34ad8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97ba78c7fc184b549caa84cf6a4f0df1', 'request_id': 'req-e42638fb-b412-4f23-995b-61e596a34ad8'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:25.472727 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-e42638fb-b412-4f23-995b-61e596a34ad8 admin admin] The token cache is being invalidate because project 97ba78c7fc184b549caa84cf6a4f0df1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:25.473028 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-e42638fb-b412-4f23-995b-61e596a34ad8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:25.473716 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-e42638fb-b412-4f23-995b-61e596a34ad8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:25.475671 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 891/1747] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:25 2022] DELETE /identity/v3/projects/97ba78c7fc184b549caa84cf6a4f0df1 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:25.481158 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8bf610-0b9d-4611-b349-a95bcc47d216 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:25.481622 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8bf610-0b9d-4611-b349-a95bcc47d216 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:25.481966 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8bf610-0b9d-4611-b349-a95bcc47d216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:25.513037 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ef8bf610-0b9d-4611-b349-a95bcc47d216 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:25.514405 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ef8bf610-0b9d-4611-b349-a95bcc47d216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:25.547069 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42638fb-b412-4f23-995b-61e596a34ad8 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:25.548634 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-e42638fb-b412-4f23-995b-61e596a34ad8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:25.620629 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 857/1748] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:25.621564 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8bf610-0b9d-4611-b349-a95bcc47d216 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:25.622939 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ef8bf610-0b9d-4611-b349-a95bcc47d216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:25.635738 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8bf610-0b9d-4611-b349-a95bcc47d216 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:25.638648 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42638fb-b412-4f23-995b-61e596a34ad8 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:25.642058 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5508e709-ac3f-401e-8c3c-35dd47ea1b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:25.642716 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508e709-ac3f-401e-8c3c-35dd47ea1b38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:25.642999 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508e709-ac3f-401e-8c3c-35dd47ea1b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:25.643277 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508e709-ac3f-401e-8c3c-35dd47ea1b38 admin admin] PATH_INFO: `/v3/projects/2cd41b6c817b4d70819489684de572cd` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:25.644441 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5508e709-ac3f-401e-8c3c-35dd47ea1b38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cd41b6c817b4d70819489684de572cd)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:25.644866 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7658a7e-140e-4073-b086-f987bb18c4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:25.645609 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7658a7e-140e-4073-b086-f987bb18c4ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:25.645962 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7658a7e-140e-4073-b086-f987bb18c4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:25.646308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7658a7e-140e-4073-b086-f987bb18c4ef admin admin] PATH_INFO: `/v3/projects/7a0c5317455e4f4a8bf6bcd53c957638` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:25.646405 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5508e709-ac3f-401e-8c3c-35dd47ea1b38 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:25.647713 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7658a7e-140e-4073-b086-f987bb18c4ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a0c5317455e4f4a8bf6bcd53c957638)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:25.650419 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7658a7e-140e-4073-b086-f987bb18c4ef admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:25.700049 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5508e709-ac3f-401e-8c3c-35dd47ea1b38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cd41b6c817b4d70819489684de572cd', 'request_id': 'req-5508e709-ac3f-401e-8c3c-35dd47ea1b38'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:25.708612 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5508e709-ac3f-401e-8c3c-35dd47ea1b38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cd41b6c817b4d70819489684de572cd', 'request_id': 'req-5508e709-ac3f-401e-8c3c-35dd47ea1b38'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:25.715572 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-e7658a7e-140e-4073-b086-f987bb18c4ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a0c5317455e4f4a8bf6bcd53c957638', 'request_id': 'req-e7658a7e-140e-4073-b086-f987bb18c4ef'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:25.716676 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5508e709-ac3f-401e-8c3c-35dd47ea1b38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cd41b6c817b4d70819489684de572cd', 'request_id': 'req-5508e709-ac3f-401e-8c3c-35dd47ea1b38'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:25.724315 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-e7658a7e-140e-4073-b086-f987bb18c4ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a0c5317455e4f4a8bf6bcd53c957638', 'request_id': 'req-e7658a7e-140e-4073-b086-f987bb18c4ef'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:25.730325 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5508e709-ac3f-401e-8c3c-35dd47ea1b38 admin admin] The token cache is being invalidate because project 2cd41b6c817b4d70819489684de572cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:25.730734 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5508e709-ac3f-401e-8c3c-35dd47ea1b38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:25.731375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5508e709-ac3f-401e-8c3c-35dd47ea1b38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:25.731475 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-e7658a7e-140e-4073-b086-f987bb18c4ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a0c5317455e4f4a8bf6bcd53c957638', 'request_id': 'req-e7658a7e-140e-4073-b086-f987bb18c4ef'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:25.733255 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 858/1749] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:25 2022] DELETE /identity/v3/projects/2cd41b6c817b4d70819489684de572cd => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:25.734303 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bfd044-f841-421e-9fc8-5066075a4454 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:25.734603 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bfd044-f841-421e-9fc8-5066075a4454 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:25.734884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bfd044-f841-421e-9fc8-5066075a4454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:25.748868 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-e7658a7e-140e-4073-b086-f987bb18c4ef admin admin] The token cache is being invalidate because project 7a0c5317455e4f4a8bf6bcd53c957638 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:25.749283 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-e7658a7e-140e-4073-b086-f987bb18c4ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:25.750520 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-e7658a7e-140e-4073-b086-f987bb18c4ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:25.752446 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 892/1750] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:25 2022] DELETE /identity/v3/projects/7a0c5317455e4f4a8bf6bcd53c957638 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:25.753395 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7658a7e-140e-4073-b086-f987bb18c4ef admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:25.754740 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-e7658a7e-140e-4073-b086-f987bb18c4ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:25.761372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-97bfd044-f841-421e-9fc8-5066075a4454 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:25.763777 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-97bfd044-f841-421e-9fc8-5066075a4454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:25.785722 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:25.817887 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:25.838325 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 859/1751] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:25.839713 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6595a-7845-4ac2-901d-2b12aae26ea8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:25.840064 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6595a-7845-4ac2-901d-2b12aae26ea8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:25.840412 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6595a-7845-4ac2-901d-2b12aae26ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:25.847954 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7658a7e-140e-4073-b086-f987bb18c4ef admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:25.855161 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a074c97e-8f3b-4d2b-aca4-f883f80967be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:25.856086 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074c97e-8f3b-4d2b-aca4-f883f80967be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:25.856584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074c97e-8f3b-4d2b-aca4-f883f80967be admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:25.857020 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074c97e-8f3b-4d2b-aca4-f883f80967be admin admin] PATH_INFO: `/v3/users/0eb53f5d7e73428a8ee8e723042fa550` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:25.858931 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a074c97e-8f3b-4d2b-aca4-f883f80967be admin admin] RBAC: Authorizing `identity:delete_user(user_id=0eb53f5d7e73428a8ee8e723042fa550)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:25.861101 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a074c97e-8f3b-4d2b-aca4-f883f80967be admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:25.863560 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d0c6595a-7845-4ac2-901d-2b12aae26ea8 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:25.864886 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d0c6595a-7845-4ac2-901d-2b12aae26ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:25.872947 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 860/1752] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:25.873745 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6595a-7845-4ac2-901d-2b12aae26ea8 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:25.885382 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6595a-7845-4ac2-901d-2b12aae26ea8 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:25.891262 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc9b3c-822e-4f2f-8112-8ffe39064674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:25.891983 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc9b3c-822e-4f2f-8112-8ffe39064674 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:25.892286 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc9b3c-822e-4f2f-8112-8ffe39064674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:25.892575 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc9b3c-822e-4f2f-8112-8ffe39064674 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:25.893517 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdc9b3c-822e-4f2f-8112-8ffe39064674 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:25.893945 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdc9b3c-822e-4f2f-8112-8ffe39064674 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:25.905047 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdc9b3c-822e-4f2f-8112-8ffe39064674 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:25.922449 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 861/1753] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:09:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:25.922449 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7d8e3-34e4-4dea-8bb5-13ff02cda2b0 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:25.922449 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7d8e3-34e4-4dea-8bb5-13ff02cda2b0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:25.922449 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7d8e3-34e4-4dea-8bb5-13ff02cda2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:25.925929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-a074c97e-8f3b-4d2b-aca4-f883f80967be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0eb53f5d7e73428a8ee8e723042fa550'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:25.926787 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-a074c97e-8f3b-4d2b-aca4-f883f80967be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0eb53f5d7e73428a8ee8e723042fa550'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:25.941925 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d6f7d8e3-34e4-4dea-8bb5-13ff02cda2b0 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:25.943208 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d6f7d8e3-34e4-4dea-8bb5-13ff02cda2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:25.943340 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-a074c97e-8f3b-4d2b-aca4-f883f80967be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0eb53f5d7e73428a8ee8e723042fa550'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:25.951775 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-a074c97e-8f3b-4d2b-aca4-f883f80967be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0eb53f5d7e73428a8ee8e723042fa550'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:25.952610 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-a074c97e-8f3b-4d2b-aca4-f883f80967be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0eb53f5d7e73428a8ee8e723042fa550'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:25.953490 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 862/1754] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:25.954284 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7d8e3-34e4-4dea-8bb5-13ff02cda2b0 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:25.964795 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7d8e3-34e4-4dea-8bb5-13ff02cda2b0 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:25.969337 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 893/1755] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:25 2022] DELETE /identity/v3/users/0eb53f5d7e73428a8ee8e723042fa550 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:25.969658 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00588444-045c-4715-90d3-ee088116d3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:25.970156 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a074c97e-8f3b-4d2b-aca4-f883f80967be admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:25.970263 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00588444-045c-4715-90d3-ee088116d3f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:25.970533 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00588444-045c-4715-90d3-ee088116d3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:25.970906 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00588444-045c-4715-90d3-ee088116d3f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:25.971567 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00588444-045c-4715-90d3-ee088116d3f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:25.971674 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-a074c97e-8f3b-4d2b-aca4-f883f80967be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:25.974367 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00588444-045c-4715-90d3-ee088116d3f6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:26.022788 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 863/1756] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:25 2022] POST /identity/v3/projects => generated 373 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:26.028446 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00588444-045c-4715-90d3-ee088116d3f6 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:26.030543 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-00588444-045c-4715-90d3-ee088116d3f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:26.047264 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a074c97e-8f3b-4d2b-aca4-f883f80967be admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:26.112995 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00588444-045c-4715-90d3-ee088116d3f6 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:26.118582 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:26.120137 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6530f852-5a91-43bb-81de-26dce5d5ab76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:26.120983 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6530f852-5a91-43bb-81de-26dce5d5ab76 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:26.121482 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6530f852-5a91-43bb-81de-26dce5d5ab76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:26.121768 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6530f852-5a91-43bb-81de-26dce5d5ab76 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:26.122730 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6530f852-5a91-43bb-81de-26dce5d5ab76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:26.125330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6530f852-5a91-43bb-81de-26dce5d5ab76 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:26.133950 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfdd738-85ce-4bed-89f9-59de27ed1086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:26.135230 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdd738-85ce-4bed-89f9-59de27ed1086 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:26.136502 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdd738-85ce-4bed-89f9-59de27ed1086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:26.136635 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdd738-85ce-4bed-89f9-59de27ed1086 admin admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:26.138222 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfdd738-85ce-4bed-89f9-59de27ed1086 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:26.138724 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfdd738-85ce-4bed-89f9-59de27ed1086 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:26.140647 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfdd738-85ce-4bed-89f9-59de27ed1086 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:26.153770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 864/1757] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:26 2022] POST /identity/v3/users => generated 287 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:09:26.155183 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 894/1758] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:09:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:26.157922 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6530f852-5a91-43bb-81de-26dce5d5ab76 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:26.159310 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfdd738-85ce-4bed-89f9-59de27ed1086 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:26.169451 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6530f852-5a91-43bb-81de-26dce5d5ab76 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:26.171471 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfdd738-85ce-4bed-89f9-59de27ed1086 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:26.174149 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940db59d-42fe-4df4-a1c5-63b4dbc29b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:26.175491 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940db59d-42fe-4df4-a1c5-63b4dbc29b44 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:26.175491 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940db59d-42fe-4df4-a1c5-63b4dbc29b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:26.175491 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940db59d-42fe-4df4-a1c5-63b4dbc29b44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:26.176423 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940db59d-42fe-4df4-a1c5-63b4dbc29b44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:26.177541 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939d059-0732-4e37-9db1-5ec651c251da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:26.178083 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939d059-0732-4e37-9db1-5ec651c251da admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:26.178378 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940db59d-42fe-4df4-a1c5-63b4dbc29b44 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:26.178585 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939d059-0732-4e37-9db1-5ec651c251da admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:26.179025 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939d059-0732-4e37-9db1-5ec651c251da admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:26.179774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1939d059-0732-4e37-9db1-5ec651c251da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:26.182658 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1939d059-0732-4e37-9db1-5ec651c251da admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:26.190393 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 865/1759] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:26.194417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940db59d-42fe-4df4-a1c5-63b4dbc29b44 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:26.207925 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940db59d-42fe-4df4-a1c5-63b4dbc29b44 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:26.213805 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076f7ee-77fb-43e4-96c9-de5408d81ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:26.214421 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076f7ee-77fb-43e4-96c9-de5408d81ebc admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:26.214704 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076f7ee-77fb-43e4-96c9-de5408d81ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:26.214979 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076f7ee-77fb-43e4-96c9-de5408d81ebc admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:26.215773 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0076f7ee-77fb-43e4-96c9-de5408d81ebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:26.216990 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 895/1760] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:26 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:26.220866 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0076f7ee-77fb-43e4-96c9-de5408d81ebc admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:26.221803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939d059-0732-4e37-9db1-5ec651c251da admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:26.240519 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 866/1761] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:26 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:26.241533 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076f7ee-77fb-43e4-96c9-de5408d81ebc admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:26.261433 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076f7ee-77fb-43e4-96c9-de5408d81ebc admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:26.307861 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939d059-0732-4e37-9db1-5ec651c251da admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:26.315102 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68d291-caf3-4f07-a75e-a49ccf1e5eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:26.316328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68d291-caf3-4f07-a75e-a49ccf1e5eb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:26.317051 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68d291-caf3-4f07-a75e-a49ccf1e5eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:26.317502 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68d291-caf3-4f07-a75e-a49ccf1e5eb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:26.318592 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e68d291-caf3-4f07-a75e-a49ccf1e5eb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:26.322134 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e68d291-caf3-4f07-a75e-a49ccf1e5eb0 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:26.331521 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f29176-f073-42b6-b0fd-cd4a698a34e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:26.332205 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f29176-f073-42b6-b0fd-cd4a698a34e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:26.332518 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f29176-f073-42b6-b0fd-cd4a698a34e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:26.332880 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f29176-f073-42b6-b0fd-cd4a698a34e9 admin admin] PATH_INFO: `/v3/projects/2832a64a94ee490fa8b297eb4c364615/users/19bbe82a21ec4833923497a3e095caef/roles/926b9f36c24847c582c84af904250abc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:26.350347 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f29176-f073-42b6-b0fd-cd4a698a34e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2832a64a94ee490fa8b297eb4c364615, user_id=19bbe82a21ec4833923497a3e095caef, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:26.355270 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 896/1762] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:26 2022] POST /identity/v3/users => generated 293 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:26.355441 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f29176-f073-42b6-b0fd-cd4a698a34e9 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:26.356574 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:26.356735 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68d291-caf3-4f07-a75e-a49ccf1e5eb0 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:26.367072 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68d291-caf3-4f07-a75e-a49ccf1e5eb0 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:26.375634 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 867/1763] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:26 2022] PUT /identity/v3/projects/2832a64a94ee490fa8b297eb4c364615/users/19bbe82a21ec4833923497a3e095caef/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:26.388944 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1308895a-abdb-474e-b37c-67b4835347f2 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:26.389356 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1308895a-abdb-474e-b37c-67b4835347f2 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:26.389939 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1308895a-abdb-474e-b37c-67b4835347f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:26.430206 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-1308895a-abdb-474e-b37c-67b4835347f2 None None] MFA Rules not processed for user `19bbe82a21ec4833923497a3e095caef`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:26.431386 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1308895a-abdb-474e-b37c-67b4835347f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:26.436921 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c412c1-5e27-4758-9d39-5b25274308e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:26.437553 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c412c1-5e27-4758-9d39-5b25274308e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:26.437834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c412c1-5e27-4758-9d39-5b25274308e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:26.438109 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c412c1-5e27-4758-9d39-5b25274308e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:26.438810 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c412c1-5e27-4758-9d39-5b25274308e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:26.440909 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c412c1-5e27-4758-9d39-5b25274308e7 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:26.451077 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 897/1764] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:26 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:26.453459 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c412c1-5e27-4758-9d39-5b25274308e7 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:26.462074 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-c3c412c1-5e27-4758-9d39-5b25274308e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:26.478212 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c412c1-5e27-4758-9d39-5b25274308e7 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:26.483143 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f78ad03-63be-4aa0-9ad1-2b34cebe0da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:26.483913 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78ad03-63be-4aa0-9ad1-2b34cebe0da4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:26.484184 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78ad03-63be-4aa0-9ad1-2b34cebe0da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:26.484689 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78ad03-63be-4aa0-9ad1-2b34cebe0da4 admin admin] PATH_INFO: `/v3/projects/833b90f7b4b747c3b5253f74535062d5` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:26.486440 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f78ad03-63be-4aa0-9ad1-2b34cebe0da4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=833b90f7b4b747c3b5253f74535062d5)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:26.489282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f78ad03-63be-4aa0-9ad1-2b34cebe0da4 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:26.502090 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 868/1765] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:26.505764 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1308895a-abdb-474e-b37c-67b4835347f2 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:26.519363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1308895a-abdb-474e-b37c-67b4835347f2 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:26.524957 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d06d6-d675-46cf-b6c8-4f0cbd4f5bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:26.525590 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d06d6-d675-46cf-b6c8-4f0cbd4f5bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:26.525875 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d06d6-d675-46cf-b6c8-4f0cbd4f5bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:26.526153 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d06d6-d675-46cf-b6c8-4f0cbd4f5bd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:26.526881 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31d06d6-d675-46cf-b6c8-4f0cbd4f5bd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:26.528543 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31d06d6-d675-46cf-b6c8-4f0cbd4f5bd8 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:26.540658 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 869/1766] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:26.540849 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:26.541210 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d06d6-d675-46cf-b6c8-4f0cbd4f5bd8 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:26.545186 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-7f78ad03-63be-4aa0-9ad1-2b34cebe0da4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '833b90f7b4b747c3b5253f74535062d5', 'request_id': 'req-7f78ad03-63be-4aa0-9ad1-2b34cebe0da4'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:26.553845 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d06d6-d675-46cf-b6c8-4f0cbd4f5bd8 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:26.556675 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-7f78ad03-63be-4aa0-9ad1-2b34cebe0da4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '833b90f7b4b747c3b5253f74535062d5', 'request_id': 'req-7f78ad03-63be-4aa0-9ad1-2b34cebe0da4'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:26.564174 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-7f78ad03-63be-4aa0-9ad1-2b34cebe0da4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '833b90f7b4b747c3b5253f74535062d5', 'request_id': 'req-7f78ad03-63be-4aa0-9ad1-2b34cebe0da4'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:26.588390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-7f78ad03-63be-4aa0-9ad1-2b34cebe0da4 admin admin] The token cache is being invalidate because project 833b90f7b4b747c3b5253f74535062d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:26.588390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-7f78ad03-63be-4aa0-9ad1-2b34cebe0da4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:26.589376 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-7f78ad03-63be-4aa0-9ad1-2b34cebe0da4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:26.595745 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 898/1767] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:26 2022] DELETE /identity/v3/projects/833b90f7b4b747c3b5253f74535062d5 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:26.596978 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f78ad03-63be-4aa0-9ad1-2b34cebe0da4 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:26.598811 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-7f78ad03-63be-4aa0-9ad1-2b34cebe0da4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:26.634106 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4775ec0-6d38-4587-ac6f-019d24f07437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:26.634846 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4775ec0-6d38-4587-ac6f-019d24f07437 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:26.635167 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4775ec0-6d38-4587-ac6f-019d24f07437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:26.635447 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4775ec0-6d38-4587-ac6f-019d24f07437 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:26.636113 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4775ec0-6d38-4587-ac6f-019d24f07437 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:26.638040 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4775ec0-6d38-4587-ac6f-019d24f07437 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:26.666471 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 870/1768] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:26 2022] POST /identity/v3/projects => generated 371 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:26.667732 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8ba0d-ffc4-4e12-8ce1-a33053d138de None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:26.668015 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8ba0d-ffc4-4e12-8ce1-a33053d138de None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:26.668294 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8ba0d-ffc4-4e12-8ce1-a33053d138de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:26.679038 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f78ad03-63be-4aa0-9ad1-2b34cebe0da4 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:26.689012 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-bce8ba0d-ffc4-4e12-8ce1-a33053d138de None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:26.690294 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-bce8ba0d-ffc4-4e12-8ce1-a33053d138de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:26.754747 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1709ae-07e8-4005-8c38-8d71976d9cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:26.755811 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1709ae-07e8-4005-8c38-8d71976d9cb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:26.756161 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1709ae-07e8-4005-8c38-8d71976d9cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:26.756441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1709ae-07e8-4005-8c38-8d71976d9cb6 admin admin] PATH_INFO: `/v3/projects/c2b52d88e4a342029e987ec9656d6246/users/1933cd659f4d4cac8c87e45581f9eae3/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:26.764647 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:26.776171 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1709ae-07e8-4005-8c38-8d71976d9cb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2b52d88e4a342029e987ec9656d6246, user_id=1933cd659f4d4cac8c87e45581f9eae3, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:26.779159 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1709ae-07e8-4005-8c38-8d71976d9cb6 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:26.786138 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 871/1769] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:26.786138 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce8ba0d-ffc4-4e12-8ce1-a33053d138de None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:26.786138 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-bce8ba0d-ffc4-4e12-8ce1-a33053d138de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:26.794595 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 899/1770] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:26 2022] PUT /identity/v3/projects/c2b52d88e4a342029e987ec9656d6246/users/1933cd659f4d4cac8c87e45581f9eae3/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:26.803297 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce8ba0d-ffc4-4e12-8ce1-a33053d138de None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:26.803416 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994503cc-20fc-46d4-a9dc-57b308643a74 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:26.803416 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994503cc-20fc-46d4-a9dc-57b308643a74 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:26.803416 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994503cc-20fc-46d4-a9dc-57b308643a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:26.819643 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-994503cc-20fc-46d4-a9dc-57b308643a74 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:26.820969 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-994503cc-20fc-46d4-a9dc-57b308643a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:26.866887 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada6b30-7391-425b-8f8b-dbe667c0d0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:26.868095 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada6b30-7391-425b-8f8b-dbe667c0d0f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:26.868344 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada6b30-7391-425b-8f8b-dbe667c0d0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:26.868683 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada6b30-7391-425b-8f8b-dbe667c0d0f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:26.869398 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ada6b30-7391-425b-8f8b-dbe667c0d0f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:26.871325 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ada6b30-7391-425b-8f8b-dbe667c0d0f4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:26.896568 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 872/1771] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:26 2022] POST /identity/v3/users => generated 292 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:26.896733 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 900/1772] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:26.897564 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65cbc9-d64a-4566-a3a2-57f2b8e06ba3 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:26.897956 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65cbc9-d64a-4566-a3a2-57f2b8e06ba3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:26.898410 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65cbc9-d64a-4566-a3a2-57f2b8e06ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:26.900710 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994503cc-20fc-46d4-a9dc-57b308643a74 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:26.905700 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:26.919820 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994503cc-20fc-46d4-a9dc-57b308643a74 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:26.925408 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cf7d01-9981-4518-b555-7eb975431ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:26.926121 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf7d01-9981-4518-b555-7eb975431ddf admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:26.926449 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf7d01-9981-4518-b555-7eb975431ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:26.926719 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf7d01-9981-4518-b555-7eb975431ddf admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:26.927677 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cf7d01-9981-4518-b555-7eb975431ddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:26.929341 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cf7d01-9981-4518-b555-7eb975431ddf admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:26.934967 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-3c65cbc9-d64a-4566-a3a2-57f2b8e06ba3 None None] MFA Rules not processed for user `1933cd659f4d4cac8c87e45581f9eae3`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:26.936218 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3c65cbc9-d64a-4566-a3a2-57f2b8e06ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:26.940418 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 901/1773] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:26.941472 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cf7d01-9981-4518-b555-7eb975431ddf admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:26.955362 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cf7d01-9981-4518-b555-7eb975431ddf admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:26.960728 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8948985e-ebd5-45bf-9496-b0cdbd236a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:26.962024 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948985e-ebd5-45bf-9496-b0cdbd236a0f admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:26.962411 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948985e-ebd5-45bf-9496-b0cdbd236a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:26.963500 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948985e-ebd5-45bf-9496-b0cdbd236a0f admin admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:26.964327 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8948985e-ebd5-45bf-9496-b0cdbd236a0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:26.964762 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8948985e-ebd5-45bf-9496-b0cdbd236a0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:26.966485 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8948985e-ebd5-45bf-9496-b0cdbd236a0f admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:26.988366 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 902/1774] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:09:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:26.988687 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8948985e-ebd5-45bf-9496-b0cdbd236a0f admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:27.000877 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8948985e-ebd5-45bf-9496-b0cdbd236a0f admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:27.005765 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580e760-6d7b-4434-8175-5caba3354c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:27.006631 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580e760-6d7b-4434-8175-5caba3354c72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:27.006915 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580e760-6d7b-4434-8175-5caba3354c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:27.007512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580e760-6d7b-4434-8175-5caba3354c72 admin admin] PATH_INFO: `/v3/projects/97ff833224a445d78f7963402a25d741/users/2a5e66c743a84515a66dcd31322cda06/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:27.024134 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 873/1775] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:27.025115 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65cbc9-d64a-4566-a3a2-57f2b8e06ba3 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:27.028984 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d580e760-6d7b-4434-8175-5caba3354c72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97ff833224a445d78f7963402a25d741, user_id=2a5e66c743a84515a66dcd31322cda06, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:27.032935 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d580e760-6d7b-4434-8175-5caba3354c72 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:27.036754 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65cbc9-d64a-4566-a3a2-57f2b8e06ba3 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:27.042835 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199b163-8eed-41da-a7a4-162aa5052af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:27.043453 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199b163-8eed-41da-a7a4-162aa5052af6 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:27.043717 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199b163-8eed-41da-a7a4-162aa5052af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:27.043994 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199b163-8eed-41da-a7a4-162aa5052af6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:27.044698 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1199b163-8eed-41da-a7a4-162aa5052af6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:27.046603 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1199b163-8eed-41da-a7a4-162aa5052af6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:27.052067 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 903/1776] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:26 2022] PUT /identity/v3/projects/97ff833224a445d78f7963402a25d741/users/2a5e66c743a84515a66dcd31322cda06/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:27.055913 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580e760-6d7b-4434-8175-5caba3354c72 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:27.080852 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 874/1777] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:27 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:27.081697 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199b163-8eed-41da-a7a4-162aa5052af6 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:27.132339 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580e760-6d7b-4434-8175-5caba3354c72 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:27.144876 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:27.150913 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199b163-8eed-41da-a7a4-162aa5052af6 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:27.157055 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56887239-6ee5-47a1-89ba-55b7544c4791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:27.157812 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56887239-6ee5-47a1-89ba-55b7544c4791 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:27.158093 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56887239-6ee5-47a1-89ba-55b7544c4791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:27.158369 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56887239-6ee5-47a1-89ba-55b7544c4791 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:27.159067 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56887239-6ee5-47a1-89ba-55b7544c4791 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:27.160681 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56887239-6ee5-47a1-89ba-55b7544c4791 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:27.171244 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 875/1778] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:27.171939 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56887239-6ee5-47a1-89ba-55b7544c4791 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:27.181266 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56887239-6ee5-47a1-89ba-55b7544c4791 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:27.185822 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7501d17-5b12-44ed-9fe7-e57368cec369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:27.186409 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7501d17-5b12-44ed-9fe7-e57368cec369 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:27.186689 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7501d17-5b12-44ed-9fe7-e57368cec369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:27.186966 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7501d17-5b12-44ed-9fe7-e57368cec369 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:27.187639 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7501d17-5b12-44ed-9fe7-e57368cec369 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:27.189620 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7501d17-5b12-44ed-9fe7-e57368cec369 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:27.192742 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daccb507-32fc-4c7f-b76e-21f50a718efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:27.193369 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daccb507-32fc-4c7f-b76e-21f50a718efd admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:27.193658 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daccb507-32fc-4c7f-b76e-21f50a718efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:27.193941 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daccb507-32fc-4c7f-b76e-21f50a718efd admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:27.194604 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daccb507-32fc-4c7f-b76e-21f50a718efd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:27.196712 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daccb507-32fc-4c7f-b76e-21f50a718efd admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:27.211898 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 876/1779] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:27 2022] POST /identity/v3/users => generated 289 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:27.212717 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7501d17-5b12-44ed-9fe7-e57368cec369 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:27.223448 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7501d17-5b12-44ed-9fe7-e57368cec369 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:27.229331 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 904/1780] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:27 2022] POST /identity/v3/projects => generated 387 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:27.229331 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daccb507-32fc-4c7f-b76e-21f50a718efd admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:27.229563 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af849a-dc0b-44b8-9128-569d69c0b823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:27.229563 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af849a-dc0b-44b8-9128-569d69c0b823 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:27.229848 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af849a-dc0b-44b8-9128-569d69c0b823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:27.230155 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af849a-dc0b-44b8-9128-569d69c0b823 admin admin] PATH_INFO: `/v3/projects/97ff833224a445d78f7963402a25d741/users/2a5e66c743a84515a66dcd31322cda06/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:27.232073 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77af849a-dc0b-44b8-9128-569d69c0b823 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97ff833224a445d78f7963402a25d741, user_id=2a5e66c743a84515a66dcd31322cda06, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:27.234742 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77af849a-dc0b-44b8-9128-569d69c0b823 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:27.246100 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 877/1781] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:27 2022] PUT /identity/v3/projects/97ff833224a445d78f7963402a25d741/users/2a5e66c743a84515a66dcd31322cda06/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:27.246979 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af849a-dc0b-44b8-9128-569d69c0b823 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:27.276799 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:27.292628 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daccb507-32fc-4c7f-b76e-21f50a718efd admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:27.318341 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af849a-dc0b-44b8-9128-569d69c0b823 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:27.323935 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b90c9d8-d95a-43cc-be25-bb10d32e9add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:27.324594 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b90c9d8-d95a-43cc-be25-bb10d32e9add admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:27.324883 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b90c9d8-d95a-43cc-be25-bb10d32e9add admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:27.325162 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b90c9d8-d95a-43cc-be25-bb10d32e9add admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:27.325840 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b90c9d8-d95a-43cc-be25-bb10d32e9add admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:27.327781 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b90c9d8-d95a-43cc-be25-bb10d32e9add admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:27.352169 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 878/1782] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:27 2022] POST /identity/v3/users => generated 300 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:27.353542 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc71dbc-b419-4182-9e73-eb3405e111a3 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:27.353828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc71dbc-b419-4182-9e73-eb3405e111a3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:27.354106 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc71dbc-b419-4182-9e73-eb3405e111a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:27.370817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb07a6-d4ca-4039-b49c-ecc2995e8a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:27.371563 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb07a6-d4ca-4039-b49c-ecc2995e8a26 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:27.371855 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb07a6-d4ca-4039-b49c-ecc2995e8a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:27.372137 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb07a6-d4ca-4039-b49c-ecc2995e8a26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:27.372885 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb07a6-d4ca-4039-b49c-ecc2995e8a26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:27.374487 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb07a6-d4ca-4039-b49c-ecc2995e8a26 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:27.387821 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 905/1783] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:27.388886 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5bc71dbc-b419-4182-9e73-eb3405e111a3 None None] MFA Rules not processed for user `2a5e66c743a84515a66dcd31322cda06`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:27.389131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb07a6-d4ca-4039-b49c-ecc2995e8a26 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:27.390121 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5bc71dbc-b419-4182-9e73-eb3405e111a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:27.405560 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb07a6-d4ca-4039-b49c-ecc2995e8a26 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:27.410762 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349140e0-0a45-48d6-afb3-b60a4aa79dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:27.411443 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349140e0-0a45-48d6-afb3-b60a4aa79dc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:27.411736 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349140e0-0a45-48d6-afb3-b60a4aa79dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:27.412194 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349140e0-0a45-48d6-afb3-b60a4aa79dc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:27.412953 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349140e0-0a45-48d6-afb3-b60a4aa79dc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:27.414707 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349140e0-0a45-48d6-afb3-b60a4aa79dc5 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:27.427379 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 906/1784] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:27.429457 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349140e0-0a45-48d6-afb3-b60a4aa79dc5 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:27.441554 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349140e0-0a45-48d6-afb3-b60a4aa79dc5 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:27.447334 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2db38f-d2dd-404d-ab72-5dbf5dbeb401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:27.449918 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2db38f-d2dd-404d-ab72-5dbf5dbeb401 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:27.450218 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2db38f-d2dd-404d-ab72-5dbf5dbeb401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:27.450816 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2db38f-d2dd-404d-ab72-5dbf5dbeb401 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:27.451210 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2db38f-d2dd-404d-ab72-5dbf5dbeb401 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:27.452998 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2db38f-d2dd-404d-ab72-5dbf5dbeb401 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:27.465823 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 907/1785] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:27.470389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2db38f-d2dd-404d-ab72-5dbf5dbeb401 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:27.480621 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2db38f-d2dd-404d-ab72-5dbf5dbeb401 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:27.485529 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5bb6e-76eb-4a77-a8ea-0679237e8607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:27.486193 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5bb6e-76eb-4a77-a8ea-0679237e8607 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:27.486478 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5bb6e-76eb-4a77-a8ea-0679237e8607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:27.486759 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5bb6e-76eb-4a77-a8ea-0679237e8607 admin admin] PATH_INFO: `/v3/projects/43f1aac6ef1b441d9e0f88112d7bb0c1/users/c473d73e6c364dfcab71c53c311b0a20/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:27.491717 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 879/1786] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:27 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:27.492635 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc71dbc-b419-4182-9e73-eb3405e111a3 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:27.502636 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db5bb6e-76eb-4a77-a8ea-0679237e8607 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43f1aac6ef1b441d9e0f88112d7bb0c1, user_id=c473d73e6c364dfcab71c53c311b0a20, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:27.503221 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc71dbc-b419-4182-9e73-eb3405e111a3 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:27.505442 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db5bb6e-76eb-4a77-a8ea-0679237e8607 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:27.507802 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1b54c-fd6b-438c-8560-3068509514c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:27.509057 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1b54c-fd6b-438c-8560-3068509514c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:27.509346 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1b54c-fd6b-438c-8560-3068509514c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:27.509631 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1b54c-fd6b-438c-8560-3068509514c8 admin admin] PATH_INFO: `/v3/projects/21e82617f6e64fafae1b98fa167be7ed/users/3407935f4ae74dc7aa63466001a42805/roles/926b9f36c24847c582c84af904250abc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:27.518807 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 908/1787] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:27 2022] PUT /identity/v3/projects/43f1aac6ef1b441d9e0f88112d7bb0c1/users/c473d73e6c364dfcab71c53c311b0a20/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:27.519782 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5bb6e-76eb-4a77-a8ea-0679237e8607 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:27.521928 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-8db5bb6e-76eb-4a77-a8ea-0679237e8607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:27.525476 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b1b54c-fd6b-438c-8560-3068509514c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21e82617f6e64fafae1b98fa167be7ed, user_id=3407935f4ae74dc7aa63466001a42805, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:27.528137 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b1b54c-fd6b-438c-8560-3068509514c8 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:27.541671 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 880/1788] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:27 2022] PUT /identity/v3/projects/21e82617f6e64fafae1b98fa167be7ed/users/3407935f4ae74dc7aa63466001a42805/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:27.550101 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1b54c-fd6b-438c-8560-3068509514c8 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:27.555267 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:27.589401 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5bb6e-76eb-4a77-a8ea-0679237e8607 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:27.609121 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1b54c-fd6b-438c-8560-3068509514c8 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:27.613793 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ada60-0a5b-4365-8ad4-ac3fb72ae3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:27.614421 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ada60-0a5b-4365-8ad4-ac3fb72ae3fd admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:27.614709 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ada60-0a5b-4365-8ad4-ac3fb72ae3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:27.614991 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ada60-0a5b-4365-8ad4-ac3fb72ae3fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:27.615694 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229ada60-0a5b-4365-8ad4-ac3fb72ae3fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:27.617876 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229ada60-0a5b-4365-8ad4-ac3fb72ae3fd admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:27.628304 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 881/1789] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:27.629598 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8402ec-e28c-4a02-84ec-f00481c9d9be None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:27.630058 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8402ec-e28c-4a02-84ec-f00481c9d9be None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:27.630219 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8402ec-e28c-4a02-84ec-f00481c9d9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:27.639863 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6746c85-d65a-4708-a6af-aef6f92f1060 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:27.640510 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6746c85-d65a-4708-a6af-aef6f92f1060 service cinder] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:27.640791 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6746c85-d65a-4708-a6af-aef6f92f1060 service cinder] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:27.641068 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6746c85-d65a-4708-a6af-aef6f92f1060 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:27.654444 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:27.659733 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-db8402ec-e28c-4a02-84ec-f00481c9d9be None None] MFA Rules not processed for user `3407935f4ae74dc7aa63466001a42805`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:27.660977 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-db8402ec-e28c-4a02-84ec-f00481c9d9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:27.714276 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6746c85-d65a-4708-a6af-aef6f92f1060 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:27.715729 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6746c85-d65a-4708-a6af-aef6f92f1060 service cinder] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:27.726106 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 909/1790] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:27 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:27.727163 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6746c85-d65a-4708-a6af-aef6f92f1060 service cinder] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:27.732244 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 882/1791] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:27.736474 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8402ec-e28c-4a02-84ec-f00481c9d9be None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:27.744729 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6746c85-d65a-4708-a6af-aef6f92f1060 service cinder] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:27.750038 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e6c97-9f2e-456e-bfe7-be17d66b475d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:27.750776 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e6c97-9f2e-456e-bfe7-be17d66b475d admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:27.751064 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e6c97-9f2e-456e-bfe7-be17d66b475d admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:27.751350 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e6c97-9f2e-456e-bfe7-be17d66b475d admin admin] PATH_INFO: `/v3/projects/43f1aac6ef1b441d9e0f88112d7bb0c1/users/c473d73e6c364dfcab71c53c311b0a20/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:27.753318 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8402ec-e28c-4a02-84ec-f00481c9d9be None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:27.753685 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4e6c97-9f2e-456e-bfe7-be17d66b475d admin admin] RBAC: Authorizing `identity:create_grant(project_id=43f1aac6ef1b441d9e0f88112d7bb0c1, user_id=c473d73e6c364dfcab71c53c311b0a20, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:27.756721 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4e6c97-9f2e-456e-bfe7-be17d66b475d admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:27.760195 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e772a52a-4643-470a-96de-f21c0590ec7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:27.761645 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772a52a-4643-470a-96de-f21c0590ec7d admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:27.762077 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772a52a-4643-470a-96de-f21c0590ec7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:27.762497 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772a52a-4643-470a-96de-f21c0590ec7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:27.763319 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e772a52a-4643-470a-96de-f21c0590ec7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:27.765316 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e772a52a-4643-470a-96de-f21c0590ec7d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:27.772793 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 910/1792] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:27 2022] PUT /identity/v3/projects/43f1aac6ef1b441d9e0f88112d7bb0c1/users/c473d73e6c364dfcab71c53c311b0a20/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:27.781436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9678b7-5784-4446-9ad5-da961fdf0624 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:27.781717 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9678b7-5784-4446-9ad5-da961fdf0624 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:27.782006 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9678b7-5784-4446-9ad5-da961fdf0624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:27.811438 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 883/1793] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:27 2022] POST /identity/v3/projects => generated 379 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:09:27.818262 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e772a52a-4643-470a-96de-f21c0590ec7d admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:27.838741 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-3b9678b7-5784-4446-9ad5-da961fdf0624 None None] MFA Rules not processed for user `c473d73e6c364dfcab71c53c311b0a20`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:27.840137 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-3b9678b7-5784-4446-9ad5-da961fdf0624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:27.898490 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e772a52a-4643-470a-96de-f21c0590ec7d admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:27.903720 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a464f9-6c5a-4e02-b0a9-62062088452a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:27.904354 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a464f9-6c5a-4e02-b0a9-62062088452a admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:27.904910 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a464f9-6c5a-4e02-b0a9-62062088452a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:27.905011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a464f9-6c5a-4e02-b0a9-62062088452a admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:27.905819 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a464f9-6c5a-4e02-b0a9-62062088452a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:27.907701 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a464f9-6c5a-4e02-b0a9-62062088452a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:27.935091 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 911/1794] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:27 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:27.940555 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 884/1795] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:27 2022] POST /identity/v3/users => generated 290 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:27.942046 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9678b7-5784-4446-9ad5-da961fdf0624 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:27.948096 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a464f9-6c5a-4e02-b0a9-62062088452a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:27.954961 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:27.956994 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9678b7-5784-4446-9ad5-da961fdf0624 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:27.963796 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61334b6-08be-470e-a48c-b3b0ff79a908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:27.964567 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61334b6-08be-470e-a48c-b3b0ff79a908 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:27.964918 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61334b6-08be-470e-a48c-b3b0ff79a908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:27.965250 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61334b6-08be-470e-a48c-b3b0ff79a908 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:27.966324 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61334b6-08be-470e-a48c-b3b0ff79a908 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:27.968190 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61334b6-08be-470e-a48c-b3b0ff79a908 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:27.986259 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 912/1796] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:27.987677 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61334b6-08be-470e-a48c-b3b0ff79a908 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:28.002561 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61334b6-08be-470e-a48c-b3b0ff79a908 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:28.014761 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f63ff96-8732-47de-be95-b70fc7948841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:28.016335 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63ff96-8732-47de-be95-b70fc7948841 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:28.016676 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63ff96-8732-47de-be95-b70fc7948841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:28.016965 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63ff96-8732-47de-be95-b70fc7948841 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:28.017848 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f63ff96-8732-47de-be95-b70fc7948841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:28.020172 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f63ff96-8732-47de-be95-b70fc7948841 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:28.040279 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 913/1797] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:28.041373 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f63ff96-8732-47de-be95-b70fc7948841 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:28.042740 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a464f9-6c5a-4e02-b0a9-62062088452a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:28.048017 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddda35a-0e2a-42f8-9e7c-9ec6a3f0a116 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:28.048856 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddda35a-0e2a-42f8-9e7c-9ec6a3f0a116 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:28.049278 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddda35a-0e2a-42f8-9e7c-9ec6a3f0a116 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:28.049724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddda35a-0e2a-42f8-9e7c-9ec6a3f0a116 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:28.052388 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f63ff96-8732-47de-be95-b70fc7948841 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:28.057988 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18cfe58-e860-4ccf-a730-b0a03d7d8447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:28.058670 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18cfe58-e860-4ccf-a730-b0a03d7d8447 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:28.058959 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18cfe58-e860-4ccf-a730-b0a03d7d8447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:28.059244 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18cfe58-e860-4ccf-a730-b0a03d7d8447 admin admin] PATH_INFO: `/v3/projects/ec59270f6d3044fd80d708e44c6b7bd4/users/f435018dcd4f4e1dab8b9abc822caf1f/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:28.062082 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddda35a-0e2a-42f8-9e7c-9ec6a3f0a116 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:28.063635 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddda35a-0e2a-42f8-9e7c-9ec6a3f0a116 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:28.076428 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18cfe58-e860-4ccf-a730-b0a03d7d8447 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec59270f6d3044fd80d708e44c6b7bd4, user_id=f435018dcd4f4e1dab8b9abc822caf1f, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:28.079156 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18cfe58-e860-4ccf-a730-b0a03d7d8447 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:28.081440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 885/1798] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:27 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:28.111698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 914/1799] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:28 2022] PUT /identity/v3/projects/ec59270f6d3044fd80d708e44c6b7bd4/users/f435018dcd4f4e1dab8b9abc822caf1f/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:28.122174 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c38c30-67f9-454e-aebd-d66dabe67ccd None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:28.122817 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c38c30-67f9-454e-aebd-d66dabe67ccd None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:28.124753 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c38c30-67f9-454e-aebd-d66dabe67ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:28.166207 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a8c38c30-67f9-454e-aebd-d66dabe67ccd None None] MFA Rules not processed for user `f435018dcd4f4e1dab8b9abc822caf1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:28.168390 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a8c38c30-67f9-454e-aebd-d66dabe67ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:28.197803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18cfe58-e860-4ccf-a730-b0a03d7d8447 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:28.254177 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:28.282556 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18cfe58-e860-4ccf-a730-b0a03d7d8447 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:28.293200 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad4ffd-d0fa-4e1a-8d60-d446b9301196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:28.294010 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad4ffd-d0fa-4e1a-8d60-d446b9301196 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:28.294514 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad4ffd-d0fa-4e1a-8d60-d446b9301196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:28.294830 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad4ffd-d0fa-4e1a-8d60-d446b9301196 admin admin] PATH_INFO: `/v3/users/1933cd659f4d4cac8c87e45581f9eae3` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:28.299366 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad4ffd-d0fa-4e1a-8d60-d446b9301196 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1933cd659f4d4cac8c87e45581f9eae3)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:28.305756 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad4ffd-d0fa-4e1a-8d60-d446b9301196 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:28.317854 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 886/1800] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:28.317854 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c38c30-67f9-454e-aebd-d66dabe67ccd None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:28.332670 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c38c30-67f9-454e-aebd-d66dabe67ccd None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:28.339794 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c35778-006e-4e7a-9c3e-dbdd74b0f931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:28.340647 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c35778-006e-4e7a-9c3e-dbdd74b0f931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:28.341187 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c35778-006e-4e7a-9c3e-dbdd74b0f931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:28.341570 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c35778-006e-4e7a-9c3e-dbdd74b0f931 admin admin] PATH_INFO: `/v3/users/19bbe82a21ec4833923497a3e095caef` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:28.344041 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c35778-006e-4e7a-9c3e-dbdd74b0f931 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19bbe82a21ec4833923497a3e095caef)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:28.346882 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c35778-006e-4e7a-9c3e-dbdd74b0f931 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:28.432927 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-75c35778-006e-4e7a-9c3e-dbdd74b0f931 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19bbe82a21ec4833923497a3e095caef'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:28.436435 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-eaad4ffd-d0fa-4e1a-8d60-d446b9301196 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1933cd659f4d4cac8c87e45581f9eae3'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:28.437780 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-eaad4ffd-d0fa-4e1a-8d60-d446b9301196 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1933cd659f4d4cac8c87e45581f9eae3'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:28.452704 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-75c35778-006e-4e7a-9c3e-dbdd74b0f931 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19bbe82a21ec4833923497a3e095caef'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:28.454231 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-75c35778-006e-4e7a-9c3e-dbdd74b0f931 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19bbe82a21ec4833923497a3e095caef'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:28.461545 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-eaad4ffd-d0fa-4e1a-8d60-d446b9301196 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1933cd659f4d4cac8c87e45581f9eae3'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:28.467826 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-75c35778-006e-4e7a-9c3e-dbdd74b0f931 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19bbe82a21ec4833923497a3e095caef'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:28.468796 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-75c35778-006e-4e7a-9c3e-dbdd74b0f931 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19bbe82a21ec4833923497a3e095caef'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:28.476041 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-eaad4ffd-d0fa-4e1a-8d60-d446b9301196 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1933cd659f4d4cac8c87e45581f9eae3'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:28.477393 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-eaad4ffd-d0fa-4e1a-8d60-d446b9301196 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1933cd659f4d4cac8c87e45581f9eae3'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:28.510392 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 887/1801] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:28 2022] DELETE /identity/v3/users/19bbe82a21ec4833923497a3e095caef => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:28.510392 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c35778-006e-4e7a-9c3e-dbdd74b0f931 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:28.510392 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-75c35778-006e-4e7a-9c3e-dbdd74b0f931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:28.526535 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 915/1802] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:28 2022] DELETE /identity/v3/users/1933cd659f4d4cac8c87e45581f9eae3 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:28.528294 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea824d2b-2a24-4830-8a31-8319c3b3c1bc None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:28.528771 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea824d2b-2a24-4830-8a31-8319c3b3c1bc None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:28.529190 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea824d2b-2a24-4830-8a31-8319c3b3c1bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:28.534853 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:28.573817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-ea824d2b-2a24-4830-8a31-8319c3b3c1bc None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:28.581309 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-ea824d2b-2a24-4830-8a31-8319c3b3c1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:28.601116 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c35778-006e-4e7a-9c3e-dbdd74b0f931 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:28.672290 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a53984-5b62-42eb-96cc-d9c0132ca667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:28.673153 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a53984-5b62-42eb-96cc-d9c0132ca667 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:28.673512 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a53984-5b62-42eb-96cc-d9c0132ca667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:28.673848 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a53984-5b62-42eb-96cc-d9c0132ca667 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:28.674828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a53984-5b62-42eb-96cc-d9c0132ca667 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:28.677325 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a53984-5b62-42eb-96cc-d9c0132ca667 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:28.716565 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 916/1803] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:28.716565 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa0ca8-b131-478b-b2fb-36c5b7bbb845 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:28.716565 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa0ca8-b131-478b-b2fb-36c5b7bbb845 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:28.716565 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa0ca8-b131-478b-b2fb-36c5b7bbb845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:28.729768 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 888/1804] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:28 2022] POST /identity/v3/projects => generated 377 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:28.731070 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a53984-5b62-42eb-96cc-d9c0132ca667 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:28.733203 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d1a53984-5b62-42eb-96cc-d9c0132ca667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:28.764603 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-12aa0ca8-b131-478b-b2fb-36c5b7bbb845 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:28.771998 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-12aa0ca8-b131-478b-b2fb-36c5b7bbb845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:28.859420 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:28.887648 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 917/1805] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:28.888993 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa0ca8-b131-478b-b2fb-36c5b7bbb845 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:28.893438 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a53984-5b62-42eb-96cc-d9c0132ca667 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:28.899648 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-12aa0ca8-b131-478b-b2fb-36c5b7bbb845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:28.901970 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc280dd-f126-4a86-86c4-cf942925c380 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:28.902628 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc280dd-f126-4a86-86c4-cf942925c380 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:28.902974 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc280dd-f126-4a86-86c4-cf942925c380 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:28.903250 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc280dd-f126-4a86-86c4-cf942925c380 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:28.917585 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa0ca8-b131-478b-b2fb-36c5b7bbb845 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:28.925059 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc280dd-f126-4a86-86c4-cf942925c380 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:28.926845 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc280dd-f126-4a86-86c4-cf942925c380 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:28.939957 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 889/1806] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:28.941423 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc280dd-f126-4a86-86c4-cf942925c380 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:28.943063 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8dc280dd-f126-4a86-86c4-cf942925c380 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:28.949026 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf8ab9-8ca1-4ef5-82d8-2611f5c9334f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:28.952552 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf8ab9-8ca1-4ef5-82d8-2611f5c9334f admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:28.952903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf8ab9-8ca1-4ef5-82d8-2611f5c9334f admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:28.953185 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf8ab9-8ca1-4ef5-82d8-2611f5c9334f admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:28.953904 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf8ab9-8ca1-4ef5-82d8-2611f5c9334f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:28.955885 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf8ab9-8ca1-4ef5-82d8-2611f5c9334f admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:28.960092 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc280dd-f126-4a86-86c4-cf942925c380 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:28.974228 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44bf1a-b6d4-4181-9e5a-637a06f44b58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:28.975351 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44bf1a-b6d4-4181-9e5a-637a06f44b58 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:28.975649 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44bf1a-b6d4-4181-9e5a-637a06f44b58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:28.975984 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44bf1a-b6d4-4181-9e5a-637a06f44b58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:28.990111 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44bf1a-b6d4-4181-9e5a-637a06f44b58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:28.994721 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44bf1a-b6d4-4181-9e5a-637a06f44b58 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:28.994892 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 918/1807] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:28 2022] POST /identity/v3/users => generated 295 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:29.000415 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf8ab9-8ca1-4ef5-82d8-2611f5c9334f admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:29.003744 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 890/1808] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:29.013930 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf8ab9-8ca1-4ef5-82d8-2611f5c9334f admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:29.019297 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c310168-6a47-442e-9072-0d4bf2acdfe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:29.019914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c310168-6a47-442e-9072-0d4bf2acdfe2 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:29.020134 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c310168-6a47-442e-9072-0d4bf2acdfe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:29.021508 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c310168-6a47-442e-9072-0d4bf2acdfe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:29.022243 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c310168-6a47-442e-9072-0d4bf2acdfe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:29.024685 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c310168-6a47-442e-9072-0d4bf2acdfe2 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:29.056777 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 919/1809] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:29.058275 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44bf1a-b6d4-4181-9e5a-637a06f44b58 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:29.078462 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44bf1a-b6d4-4181-9e5a-637a06f44b58 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:29.094720 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ada92-215b-4e40-9b47-08edf6e1588f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:29.094720 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ada92-215b-4e40-9b47-08edf6e1588f admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:29.095070 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ada92-215b-4e40-9b47-08edf6e1588f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:29.095070 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ada92-215b-4e40-9b47-08edf6e1588f admin admin] PATH_INFO: `/v3/projects/2593a76a19dc4fbd9eb1f09a21d400b6/users/a8e7c83862b44dfd89a1c21fafbf4caa/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:29.136332 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ada92-215b-4e40-9b47-08edf6e1588f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2593a76a19dc4fbd9eb1f09a21d400b6, user_id=a8e7c83862b44dfd89a1c21fafbf4caa, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:29.140202 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ada92-215b-4e40-9b47-08edf6e1588f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:29.186654 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 891/1810] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:29 2022] PUT /identity/v3/projects/2593a76a19dc4fbd9eb1f09a21d400b6/users/a8e7c83862b44dfd89a1c21fafbf4caa/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:29.191032 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c310168-6a47-442e-9072-0d4bf2acdfe2 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:29.206543 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c310168-6a47-442e-9072-0d4bf2acdfe2 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:29.363786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f5526-7638-47dc-984e-194d8452e822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:29.364437 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f5526-7638-47dc-984e-194d8452e822 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:29.365102 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f5526-7638-47dc-984e-194d8452e822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:29.365218 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f5526-7638-47dc-984e-194d8452e822 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:29.366067 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f5526-7638-47dc-984e-194d8452e822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:29.368358 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f5526-7638-47dc-984e-194d8452e822 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:29.395320 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 920/1811] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:29.397868 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ada92-215b-4e40-9b47-08edf6e1588f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:29.422439 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ada92-215b-4e40-9b47-08edf6e1588f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:29.433554 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312b7e0-e0fb-4ffa-b5ce-936ee6ee50a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:29.435191 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312b7e0-e0fb-4ffa-b5ce-936ee6ee50a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:29.437694 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312b7e0-e0fb-4ffa-b5ce-936ee6ee50a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:29.439294 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312b7e0-e0fb-4ffa-b5ce-936ee6ee50a3 admin admin] PATH_INFO: `/v3/projects/2593a76a19dc4fbd9eb1f09a21d400b6/users/a8e7c83862b44dfd89a1c21fafbf4caa/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:29.447273 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d312b7e0-e0fb-4ffa-b5ce-936ee6ee50a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2593a76a19dc4fbd9eb1f09a21d400b6, user_id=a8e7c83862b44dfd89a1c21fafbf4caa, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:29.455614 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d312b7e0-e0fb-4ffa-b5ce-936ee6ee50a3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:29.459389 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:29.506029 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 892/1812] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:29 2022] PUT /identity/v3/projects/2593a76a19dc4fbd9eb1f09a21d400b6/users/a8e7c83862b44dfd89a1c21fafbf4caa/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:29.520535 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7292f7-c660-44ba-9836-039ad73f772f None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:29.522448 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7292f7-c660-44ba-9836-039ad73f772f None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:29.522911 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7292f7-c660-44ba-9836-039ad73f772f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:29.570841 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-ab7292f7-c660-44ba-9836-039ad73f772f None None] MFA Rules not processed for user `a8e7c83862b44dfd89a1c21fafbf4caa`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:29.572893 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-ab7292f7-c660-44ba-9836-039ad73f772f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:29.702392 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 921/1813] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:29 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:29.720969 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312b7e0-e0fb-4ffa-b5ce-936ee6ee50a3 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:29.722971 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d312b7e0-e0fb-4ffa-b5ce-936ee6ee50a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:29.891065 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312b7e0-e0fb-4ffa-b5ce-936ee6ee50a3 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:29.896823 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda5c24-ef6b-4a22-af71-051c27bbea02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:29.898139 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda5c24-ef6b-4a22-af71-051c27bbea02 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:29.898867 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda5c24-ef6b-4a22-af71-051c27bbea02 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:29.899660 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda5c24-ef6b-4a22-af71-051c27bbea02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:29.911533 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda5c24-ef6b-4a22-af71-051c27bbea02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:29.913818 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda5c24-ef6b-4a22-af71-051c27bbea02 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:29.926416 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 893/1814] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:29 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:29.976338 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7292f7-c660-44ba-9836-039ad73f772f None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:29.977963 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-ab7292f7-c660-44ba-9836-039ad73f772f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:29.990496 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda5c24-ef6b-4a22-af71-051c27bbea02 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:29.993341 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2eda5c24-ef6b-4a22-af71-051c27bbea02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:30.015058 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda5c24-ef6b-4a22-af71-051c27bbea02 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:30.030582 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e7cd6f-636c-4011-9e90-719ad9cc89b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:30.031611 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e7cd6f-636c-4011-9e90-719ad9cc89b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:30.032120 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e7cd6f-636c-4011-9e90-719ad9cc89b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:30.032893 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e7cd6f-636c-4011-9e90-719ad9cc89b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:30.037016 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-38e7cd6f-636c-4011-9e90-719ad9cc89b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:30.086433 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:30.125820 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7292f7-c660-44ba-9836-039ad73f772f None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:30.134392 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d020b5ab-e844-42bd-9b3d-2346242c1a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:30.135656 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020b5ab-e844-42bd-9b3d-2346242c1a71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:30.136707 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020b5ab-e844-42bd-9b3d-2346242c1a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:30.137286 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020b5ab-e844-42bd-9b3d-2346242c1a71 admin admin] PATH_INFO: `/v3/projects/c2b52d88e4a342029e987ec9656d6246` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:30.139663 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d020b5ab-e844-42bd-9b3d-2346242c1a71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2b52d88e4a342029e987ec9656d6246)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:30.141933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d020b5ab-e844-42bd-9b3d-2346242c1a71 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:30.160679 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e7cd6f-636c-4011-9e90-719ad9cc89b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:30.162168 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e7cd6f-636c-4011-9e90-719ad9cc89b6 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:30.204317 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 894/1815] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:29 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:09:30.252230 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d020b5ab-e844-42bd-9b3d-2346242c1a71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2b52d88e4a342029e987ec9656d6246', 'request_id': 'req-d020b5ab-e844-42bd-9b3d-2346242c1a71'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:30.262487 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d020b5ab-e844-42bd-9b3d-2346242c1a71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2b52d88e4a342029e987ec9656d6246', 'request_id': 'req-d020b5ab-e844-42bd-9b3d-2346242c1a71'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:30.291770 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d020b5ab-e844-42bd-9b3d-2346242c1a71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2b52d88e4a342029e987ec9656d6246', 'request_id': 'req-d020b5ab-e844-42bd-9b3d-2346242c1a71'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:30.323143 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d020b5ab-e844-42bd-9b3d-2346242c1a71 admin admin] The token cache is being invalidate because project c2b52d88e4a342029e987ec9656d6246 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:30.324717 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d020b5ab-e844-42bd-9b3d-2346242c1a71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:30.325484 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d020b5ab-e844-42bd-9b3d-2346242c1a71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:30.339634 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 922/1816] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:29 2022] DELETE /identity/v3/projects/c2b52d88e4a342029e987ec9656d6246 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:30.340880 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e7cd6f-636c-4011-9e90-719ad9cc89b6 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:30.361286 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e7cd6f-636c-4011-9e90-719ad9cc89b6 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:30.467236 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe1ba42-5e5c-426c-9a99-cc00b7356b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:30.467815 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1ba42-5e5c-426c-9a99-cc00b7356b8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:30.468510 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1ba42-5e5c-426c-9a99-cc00b7356b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:30.468744 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1ba42-5e5c-426c-9a99-cc00b7356b8f admin admin] PATH_INFO: `/v3/users/c473d73e6c364dfcab71c53c311b0a20` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:30.470579 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe1ba42-5e5c-426c-9a99-cc00b7356b8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c473d73e6c364dfcab71c53c311b0a20)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:30.474210 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe1ba42-5e5c-426c-9a99-cc00b7356b8f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:30.589027 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d020b5ab-e844-42bd-9b3d-2346242c1a71 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:30.591564 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-d020b5ab-e844-42bd-9b3d-2346242c1a71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:30.618902 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:INSERT=1,keystone:SELECT=94,keystone:DELETE=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:30.637578 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5fe1ba42-5e5c-426c-9a99-cc00b7356b8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c473d73e6c364dfcab71c53c311b0a20'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:30.645996 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d020b5ab-e844-42bd-9b3d-2346242c1a71 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:30.660573 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e9e793-04f1-4c18-8bf1-0080e22d67ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:30.662176 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9e793-04f1-4c18-8bf1-0080e22d67ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:30.662463 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9e793-04f1-4c18-8bf1-0080e22d67ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:30.662751 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9e793-04f1-4c18-8bf1-0080e22d67ca admin admin] PATH_INFO: `/v3/projects/2832a64a94ee490fa8b297eb4c364615` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:30.666977 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e9e793-04f1-4c18-8bf1-0080e22d67ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=2832a64a94ee490fa8b297eb4c364615)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:30.669372 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e9e793-04f1-4c18-8bf1-0080e22d67ca admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:30.674797 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5fe1ba42-5e5c-426c-9a99-cc00b7356b8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c473d73e6c364dfcab71c53c311b0a20'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:30.679195 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5fe1ba42-5e5c-426c-9a99-cc00b7356b8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c473d73e6c364dfcab71c53c311b0a20'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:30.711513 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5fe1ba42-5e5c-426c-9a99-cc00b7356b8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c473d73e6c364dfcab71c53c311b0a20'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:30.713051 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5fe1ba42-5e5c-426c-9a99-cc00b7356b8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c473d73e6c364dfcab71c53c311b0a20'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:30.780801 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 895/1817] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:30 2022] DELETE /identity/v3/users/c473d73e6c364dfcab71c53c311b0a20 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:30.828023 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-f2e9e793-04f1-4c18-8bf1-0080e22d67ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2832a64a94ee490fa8b297eb4c364615', 'request_id': 'req-f2e9e793-04f1-4c18-8bf1-0080e22d67ca'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:30.845638 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-f2e9e793-04f1-4c18-8bf1-0080e22d67ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2832a64a94ee490fa8b297eb4c364615', 'request_id': 'req-f2e9e793-04f1-4c18-8bf1-0080e22d67ca'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:30.863351 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-f2e9e793-04f1-4c18-8bf1-0080e22d67ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2832a64a94ee490fa8b297eb4c364615', 'request_id': 'req-f2e9e793-04f1-4c18-8bf1-0080e22d67ca'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:30.889894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-f2e9e793-04f1-4c18-8bf1-0080e22d67ca admin admin] The token cache is being invalidate because project 2832a64a94ee490fa8b297eb4c364615 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:30.890250 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-f2e9e793-04f1-4c18-8bf1-0080e22d67ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:30.891746 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-f2e9e793-04f1-4c18-8bf1-0080e22d67ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:30.894494 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 923/1818] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:30 2022] DELETE /identity/v3/projects/2832a64a94ee490fa8b297eb4c364615 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:30.898773 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe1ba42-5e5c-426c-9a99-cc00b7356b8f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:30.900470 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5fe1ba42-5e5c-426c-9a99-cc00b7356b8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:31.138573 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe1ba42-5e5c-426c-9a99-cc00b7356b8f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:31.339173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba7f92-cd42-470a-bd3a-f0f5b5ec63d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:31.340065 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba7f92-cd42-470a-bd3a-f0f5b5ec63d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:31.340586 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba7f92-cd42-470a-bd3a-f0f5b5ec63d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:31.341055 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba7f92-cd42-470a-bd3a-f0f5b5ec63d7 admin admin] PATH_INFO: `/v3/users/2a5e66c743a84515a66dcd31322cda06` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:31.346656 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dba7f92-cd42-470a-bd3a-f0f5b5ec63d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a5e66c743a84515a66dcd31322cda06)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:31.348844 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dba7f92-cd42-470a-bd3a-f0f5b5ec63d7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:31.437381 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4dba7f92-cd42-470a-bd3a-f0f5b5ec63d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a5e66c743a84515a66dcd31322cda06'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:31.457838 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4dba7f92-cd42-470a-bd3a-f0f5b5ec63d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a5e66c743a84515a66dcd31322cda06'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:31.458984 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4dba7f92-cd42-470a-bd3a-f0f5b5ec63d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a5e66c743a84515a66dcd31322cda06'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:31.476211 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4dba7f92-cd42-470a-bd3a-f0f5b5ec63d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a5e66c743a84515a66dcd31322cda06'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:31.478456 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4dba7f92-cd42-470a-bd3a-f0f5b5ec63d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a5e66c743a84515a66dcd31322cda06'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:31.480987 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:31.545103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 896/1819] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:30 2022] DELETE /identity/v3/users/2a5e66c743a84515a66dcd31322cda06 => generated 0 bytes in 647 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:32.572686 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e9e793-04f1-4c18-8bf1-0080e22d67ca admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:32.575079 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-f2e9e793-04f1-4c18-8bf1-0080e22d67ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:32.596170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e9e793-04f1-4c18-8bf1-0080e22d67ca admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:32.605742 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba7f92-cd42-470a-bd3a-f0f5b5ec63d7 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:32.610313 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4dba7f92-cd42-470a-bd3a-f0f5b5ec63d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:32.687490 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba7f92-cd42-470a-bd3a-f0f5b5ec63d7 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:32.690884 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45810691-4964-46fe-8f32-1aa0ff5d7a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:32.691543 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45810691-4964-46fe-8f32-1aa0ff5d7a6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:32.691882 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45810691-4964-46fe-8f32-1aa0ff5d7a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:32.692166 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45810691-4964-46fe-8f32-1aa0ff5d7a6e admin admin] PATH_INFO: `/v3/projects/43f1aac6ef1b441d9e0f88112d7bb0c1` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:32.692327 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdeea2e-b4da-47fe-9f22-18b78193a858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:32.693066 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdeea2e-b4da-47fe-9f22-18b78193a858 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:32.693360 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdeea2e-b4da-47fe-9f22-18b78193a858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:32.693456 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45810691-4964-46fe-8f32-1aa0ff5d7a6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=43f1aac6ef1b441d9e0f88112d7bb0c1)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:32.693643 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdeea2e-b4da-47fe-9f22-18b78193a858 admin admin] PATH_INFO: `/v3/projects/97ff833224a445d78f7963402a25d741` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:32.694815 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdeea2e-b4da-47fe-9f22-18b78193a858 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97ff833224a445d78f7963402a25d741)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:32.695390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45810691-4964-46fe-8f32-1aa0ff5d7a6e admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:32.696917 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdeea2e-b4da-47fe-9f22-18b78193a858 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:32.697838 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:32.760232 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9bdeea2e-b4da-47fe-9f22-18b78193a858 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97ff833224a445d78f7963402a25d741', 'request_id': 'req-9bdeea2e-b4da-47fe-9f22-18b78193a858'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:32.761834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-45810691-4964-46fe-8f32-1aa0ff5d7a6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43f1aac6ef1b441d9e0f88112d7bb0c1', 'request_id': 'req-45810691-4964-46fe-8f32-1aa0ff5d7a6e'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:32.768023 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9bdeea2e-b4da-47fe-9f22-18b78193a858 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97ff833224a445d78f7963402a25d741', 'request_id': 'req-9bdeea2e-b4da-47fe-9f22-18b78193a858'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:32.769390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-45810691-4964-46fe-8f32-1aa0ff5d7a6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43f1aac6ef1b441d9e0f88112d7bb0c1', 'request_id': 'req-45810691-4964-46fe-8f32-1aa0ff5d7a6e'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:32.775034 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-45810691-4964-46fe-8f32-1aa0ff5d7a6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43f1aac6ef1b441d9e0f88112d7bb0c1', 'request_id': 'req-45810691-4964-46fe-8f32-1aa0ff5d7a6e'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:32.775175 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9bdeea2e-b4da-47fe-9f22-18b78193a858 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97ff833224a445d78f7963402a25d741', 'request_id': 'req-9bdeea2e-b4da-47fe-9f22-18b78193a858'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:32.786063 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9bdeea2e-b4da-47fe-9f22-18b78193a858 admin admin] The token cache is being invalidate because project 97ff833224a445d78f7963402a25d741 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:32.786360 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9bdeea2e-b4da-47fe-9f22-18b78193a858 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:32.787003 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9bdeea2e-b4da-47fe-9f22-18b78193a858 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:32.787301 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-45810691-4964-46fe-8f32-1aa0ff5d7a6e admin admin] The token cache is being invalidate because project 43f1aac6ef1b441d9e0f88112d7bb0c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:32.787594 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-45810691-4964-46fe-8f32-1aa0ff5d7a6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:32.788253 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-45810691-4964-46fe-8f32-1aa0ff5d7a6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:32.789303 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 897/1820] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:32 2022] DELETE /identity/v3/projects/97ff833224a445d78f7963402a25d741 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:32.796770 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 924/1821] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:32 2022] DELETE /identity/v3/projects/43f1aac6ef1b441d9e0f88112d7bb0c1 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:32.798172 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886fad4d-02ed-4740-b49f-43c69d697612 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:32.798559 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886fad4d-02ed-4740-b49f-43c69d697612 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:32.798841 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886fad4d-02ed-4740-b49f-43c69d697612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:32.805194 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c78200-fb0e-49e5-98e8-709e3f4ef492 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:32.805194 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c78200-fb0e-49e5-98e8-709e3f4ef492 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:32.805194 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c78200-fb0e-49e5-98e8-709e3f4ef492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:32.820548 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-886fad4d-02ed-4740-b49f-43c69d697612 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:32.821624 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-886fad4d-02ed-4740-b49f-43c69d697612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:32.832088 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-24c78200-fb0e-49e5-98e8-709e3f4ef492 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:32.833722 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-24c78200-fb0e-49e5-98e8-709e3f4ef492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:32.901796 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 925/1822] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:32.906658 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 898/1823] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:32.909989 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9765494-8444-4778-aee6-7ef92f35a140 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:32.910589 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9765494-8444-4778-aee6-7ef92f35a140 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:32.911010 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9765494-8444-4778-aee6-7ef92f35a140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:32.916822 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623d6d8-4a38-4b04-ad5d-ef120738f7c7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:32.917137 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623d6d8-4a38-4b04-ad5d-ef120738f7c7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:32.917418 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623d6d8-4a38-4b04-ad5d-ef120738f7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:32.938975 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4623d6d8-4a38-4b04-ad5d-ef120738f7c7 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:32.940220 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4623d6d8-4a38-4b04-ad5d-ef120738f7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:32.944948 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-a9765494-8444-4778-aee6-7ef92f35a140 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:32.946818 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-a9765494-8444-4778-aee6-7ef92f35a140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:32.956258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 899/1824] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:32.957923 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4623d6d8-4a38-4b04-ad5d-ef120738f7c7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:32.962763 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 926/1825] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:32.966632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:32.968406 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9765494-8444-4778-aee6-7ef92f35a140 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:32.983113 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4623d6d8-4a38-4b04-ad5d-ef120738f7c7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:32.986024 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9765494-8444-4778-aee6-7ef92f35a140 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:32.989905 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c052e-005a-4b7b-899e-963fd9844819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:32.990752 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c052e-005a-4b7b-899e-963fd9844819 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:32.991036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c052e-005a-4b7b-899e-963fd9844819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:32.991341 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c052e-005a-4b7b-899e-963fd9844819 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:32.992655 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47c052e-005a-4b7b-899e-963fd9844819 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:32.993075 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47c052e-005a-4b7b-899e-963fd9844819 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:32.993593 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a658ffd-b113-49cf-8b20-5d2359e8f9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:32.994297 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a658ffd-b113-49cf-8b20-5d2359e8f9a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:32.994651 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a658ffd-b113-49cf-8b20-5d2359e8f9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:32.994757 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47c052e-005a-4b7b-899e-963fd9844819 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:32.995044 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a658ffd-b113-49cf-8b20-5d2359e8f9a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:32.995844 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a658ffd-b113-49cf-8b20-5d2359e8f9a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:32.996188 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a658ffd-b113-49cf-8b20-5d2359e8f9a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:32.997900 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a658ffd-b113-49cf-8b20-5d2359e8f9a1 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:33.008844 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 900/1826] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:09:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:33.014740 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 927/1827] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:09:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:09:33.014911 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c052e-005a-4b7b-899e-963fd9844819 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:33.015785 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a658ffd-b113-49cf-8b20-5d2359e8f9a1 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:33.024036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c052e-005a-4b7b-899e-963fd9844819 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:33.026748 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a658ffd-b113-49cf-8b20-5d2359e8f9a1 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:33.029347 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce9d1d-0bc2-41da-8a20-ae0cee3d1083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:33.029937 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce9d1d-0bc2-41da-8a20-ae0cee3d1083 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:33.030290 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce9d1d-0bc2-41da-8a20-ae0cee3d1083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:33.030639 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce9d1d-0bc2-41da-8a20-ae0cee3d1083 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:33.031375 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114b12b-20c3-4ee3-bdeb-21adc2d1fc53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:33.031489 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ce9d1d-0bc2-41da-8a20-ae0cee3d1083 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:33.031992 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114b12b-20c3-4ee3-bdeb-21adc2d1fc53 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:33.032273 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114b12b-20c3-4ee3-bdeb-21adc2d1fc53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:33.032612 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114b12b-20c3-4ee3-bdeb-21adc2d1fc53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:33.033419 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114b12b-20c3-4ee3-bdeb-21adc2d1fc53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:33.033614 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ce9d1d-0bc2-41da-8a20-ae0cee3d1083 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:33.035339 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114b12b-20c3-4ee3-bdeb-21adc2d1fc53 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:33.064561 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 901/1828] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:33 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:33.068577 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce9d1d-0bc2-41da-8a20-ae0cee3d1083 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:33.069407 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 928/1829] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:33 2022] POST /identity/v3/projects => generated 391 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:33.073902 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114b12b-20c3-4ee3-bdeb-21adc2d1fc53 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:33.123296 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:33.133323 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce9d1d-0bc2-41da-8a20-ae0cee3d1083 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:33.137046 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114b12b-20c3-4ee3-bdeb-21adc2d1fc53 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:33.138787 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad07a10-b1a1-48b0-adca-0757104a2d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:33.139414 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad07a10-b1a1-48b0-adca-0757104a2d63 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:33.139697 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad07a10-b1a1-48b0-adca-0757104a2d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:33.139977 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad07a10-b1a1-48b0-adca-0757104a2d63 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:33.140729 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad07a10-b1a1-48b0-adca-0757104a2d63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:33.142451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3581d984-ecbd-40a1-b7d4-3addb3acc388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:33.142622 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad07a10-b1a1-48b0-adca-0757104a2d63 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:33.143086 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581d984-ecbd-40a1-b7d4-3addb3acc388 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:33.143377 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581d984-ecbd-40a1-b7d4-3addb3acc388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:33.143657 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581d984-ecbd-40a1-b7d4-3addb3acc388 admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:33.144324 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3581d984-ecbd-40a1-b7d4-3addb3acc388 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:33.146471 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3581d984-ecbd-40a1-b7d4-3addb3acc388 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:33.164399 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 902/1830] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:33 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:09:33.167420 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 929/1831] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:33 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:09:33.169454 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad07a10-b1a1-48b0-adca-0757104a2d63 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:33.171071 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3581d984-ecbd-40a1-b7d4-3addb3acc388 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:33.180474 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad07a10-b1a1-48b0-adca-0757104a2d63 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:33.181521 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3581d984-ecbd-40a1-b7d4-3addb3acc388 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:33.185957 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ee9e4b-cda2-4bfa-8453-7208b6e9b805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:33.186499 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7b769-da34-49a9-a3a4-cee45d9a8e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:33.186626 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee9e4b-cda2-4bfa-8453-7208b6e9b805 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:33.186837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee9e4b-cda2-4bfa-8453-7208b6e9b805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:33.187116 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7b769-da34-49a9-a3a4-cee45d9a8e46 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:33.187242 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee9e4b-cda2-4bfa-8453-7208b6e9b805 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:33.187397 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7b769-da34-49a9-a3a4-cee45d9a8e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:33.187679 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7b769-da34-49a9-a3a4-cee45d9a8e46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:33.187784 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ee9e4b-cda2-4bfa-8453-7208b6e9b805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:33.188374 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf7b769-da34-49a9-a3a4-cee45d9a8e46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:33.189403 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ee9e4b-cda2-4bfa-8453-7208b6e9b805 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:33.190031 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf7b769-da34-49a9-a3a4-cee45d9a8e46 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:33.199940 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 903/1832] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:33.201050 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 930/1833] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:33.203885 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ee9e4b-cda2-4bfa-8453-7208b6e9b805 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:33.205265 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7b769-da34-49a9-a3a4-cee45d9a8e46 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:33.213779 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ee9e4b-cda2-4bfa-8453-7208b6e9b805 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:33.214885 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7b769-da34-49a9-a3a4-cee45d9a8e46 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:33.218978 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5516636-0564-4b91-b45f-3e4ec0bcbadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:33.219588 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5516636-0564-4b91-b45f-3e4ec0bcbadb admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:33.219702 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcd70ef-e9ad-4b3a-b4ad-4626dfc5bc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:33.219877 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5516636-0564-4b91-b45f-3e4ec0bcbadb admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:33.220159 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5516636-0564-4b91-b45f-3e4ec0bcbadb admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:33.220256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd70ef-e9ad-4b3a-b4ad-4626dfc5bc8c admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:33.220505 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd70ef-e9ad-4b3a-b4ad-4626dfc5bc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:33.220804 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd70ef-e9ad-4b3a-b4ad-4626dfc5bc8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:33.220946 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5516636-0564-4b91-b45f-3e4ec0bcbadb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:33.221492 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcd70ef-e9ad-4b3a-b4ad-4626dfc5bc8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:33.222623 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5516636-0564-4b91-b45f-3e4ec0bcbadb admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:33.223055 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcd70ef-e9ad-4b3a-b4ad-4626dfc5bc8c admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:33.237331 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 931/1834] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:33.237721 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 904/1835] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:33.238488 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcd70ef-e9ad-4b3a-b4ad-4626dfc5bc8c admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:33.238712 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5516636-0564-4b91-b45f-3e4ec0bcbadb admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:33.249461 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcd70ef-e9ad-4b3a-b4ad-4626dfc5bc8c admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:33.249590 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5516636-0564-4b91-b45f-3e4ec0bcbadb admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:33.254932 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce30d5-24ad-4ae8-bed0-7c455a8bdf0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:33.255046 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc802d-c42d-454d-b885-a94a9350bf7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:33.255775 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc802d-c42d-454d-b885-a94a9350bf7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:33.256050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce30d5-24ad-4ae8-bed0-7c455a8bdf0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:33.256152 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc802d-c42d-454d-b885-a94a9350bf7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:33.256320 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc802d-c42d-454d-b885-a94a9350bf7f admin admin] PATH_INFO: `/v3/projects/77b5b204348840febd02752d62ca153e/users/37238a9211c347c48a2b4a4f2eff69c4/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:33.256398 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce30d5-24ad-4ae8-bed0-7c455a8bdf0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:33.256763 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce30d5-24ad-4ae8-bed0-7c455a8bdf0a admin admin] PATH_INFO: `/v3/projects/cf9294f3d32640dbaa215da7172d3c52/users/9a3e541656b342698f0efb9fe1ee0e07/roles/926b9f36c24847c582c84af904250abc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:33.273871 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ce30d5-24ad-4ae8-bed0-7c455a8bdf0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf9294f3d32640dbaa215da7172d3c52, user_id=9a3e541656b342698f0efb9fe1ee0e07, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:33.274092 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc802d-c42d-454d-b885-a94a9350bf7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=77b5b204348840febd02752d62ca153e, user_id=37238a9211c347c48a2b4a4f2eff69c4, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:33.276739 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ce30d5-24ad-4ae8-bed0-7c455a8bdf0a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:33.277296 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc802d-c42d-454d-b885-a94a9350bf7f admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:33.293535 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 905/1836] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:33 2022] PUT /identity/v3/projects/cf9294f3d32640dbaa215da7172d3c52/users/9a3e541656b342698f0efb9fe1ee0e07/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:33.294446 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 932/1837] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:33 2022] PUT /identity/v3/projects/77b5b204348840febd02752d62ca153e/users/37238a9211c347c48a2b4a4f2eff69c4/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:33.305447 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b558b7-7228-487f-a10d-c268a691e07c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:33.305812 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b558b7-7228-487f-a10d-c268a691e07c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:33.306169 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b558b7-7228-487f-a10d-c268a691e07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:33.306313 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b596d-877a-4f16-bdec-93783a28fc79 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:33.306858 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b596d-877a-4f16-bdec-93783a28fc79 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:33.307258 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b596d-877a-4f16-bdec-93783a28fc79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:33.336725 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d5b558b7-7228-487f-a10d-c268a691e07c None None] MFA Rules not processed for user `9a3e541656b342698f0efb9fe1ee0e07`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:33.337912 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d5b558b7-7228-487f-a10d-c268a691e07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:33.346983 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-a65b596d-877a-4f16-bdec-93783a28fc79 None None] MFA Rules not processed for user `37238a9211c347c48a2b4a4f2eff69c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:33.349114 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-a65b596d-877a-4f16-bdec-93783a28fc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:33.439084 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 906/1838] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:33.440649 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 933/1839] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:33.440812 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b558b7-7228-487f-a10d-c268a691e07c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:33.441764 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65b596d-877a-4f16-bdec-93783a28fc79 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:33.445661 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:33.518472 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65b596d-877a-4f16-bdec-93783a28fc79 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:33.523944 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11d692-1bef-419c-ac40-e3e016e9fa8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:33.525198 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11d692-1bef-419c-ac40-e3e016e9fa8e admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:33.525198 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11d692-1bef-419c-ac40-e3e016e9fa8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:33.525198 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11d692-1bef-419c-ac40-e3e016e9fa8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:33.525837 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd11d692-1bef-419c-ac40-e3e016e9fa8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:33.527743 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd11d692-1bef-419c-ac40-e3e016e9fa8e admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:33.531492 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b558b7-7228-487f-a10d-c268a691e07c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:33.539010 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cba330-4ce4-48a8-b1cc-70d9a904442a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:33.539637 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cba330-4ce4-48a8-b1cc-70d9a904442a admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:33.539926 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cba330-4ce4-48a8-b1cc-70d9a904442a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:33.540210 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cba330-4ce4-48a8-b1cc-70d9a904442a admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:33.541003 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cba330-4ce4-48a8-b1cc-70d9a904442a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:33.542992 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cba330-4ce4-48a8-b1cc-70d9a904442a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:33.566749 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 934/1840] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:33 2022] POST /identity/v3/projects => generated 393 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:33.567758 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11d692-1bef-419c-ac40-e3e016e9fa8e admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:33.573740 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 907/1841] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:33 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:33.574902 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cba330-4ce4-48a8-b1cc-70d9a904442a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:33.580412 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:33.634931 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11d692-1bef-419c-ac40-e3e016e9fa8e admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:33.637576 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cba330-4ce4-48a8-b1cc-70d9a904442a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:33.639511 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7410a-4edd-4456-9c31-2cfc4c8e07d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:33.640141 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7410a-4edd-4456-9c31-2cfc4c8e07d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:33.640429 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7410a-4edd-4456-9c31-2cfc4c8e07d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:33.640809 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7410a-4edd-4456-9c31-2cfc4c8e07d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:33.641451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7410a-4edd-4456-9c31-2cfc4c8e07d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:33.642276 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c00341-8a22-432c-9d30-14ead6dde762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:33.642887 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c00341-8a22-432c-9d30-14ead6dde762 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:33.643170 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c00341-8a22-432c-9d30-14ead6dde762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:33.643415 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7410a-4edd-4456-9c31-2cfc4c8e07d8 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:33.643502 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c00341-8a22-432c-9d30-14ead6dde762 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:33.644178 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c00341-8a22-432c-9d30-14ead6dde762 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:33.646642 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c00341-8a22-432c-9d30-14ead6dde762 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:33.673642 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 908/1842] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:33 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:09:33.680965 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c00341-8a22-432c-9d30-14ead6dde762 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:33.686337 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 935/1843] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:33 2022] POST /identity/v3/users => generated 303 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:33.693596 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7410a-4edd-4456-9c31-2cfc4c8e07d8 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:33.694846 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c00341-8a22-432c-9d30-14ead6dde762 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:33.700310 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a64b7e-8f18-4dab-82d8-7fb31b4bf509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:33.701226 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a64b7e-8f18-4dab-82d8-7fb31b4bf509 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:33.701471 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a64b7e-8f18-4dab-82d8-7fb31b4bf509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:33.701573 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a64b7e-8f18-4dab-82d8-7fb31b4bf509 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:33.702229 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a64b7e-8f18-4dab-82d8-7fb31b4bf509 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:33.703984 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a64b7e-8f18-4dab-82d8-7fb31b4bf509 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:33.705191 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7410a-4edd-4456-9c31-2cfc4c8e07d8 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:33.712336 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ed269-08f3-4a63-8031-51ee1ed636f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:33.713013 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ed269-08f3-4a63-8031-51ee1ed636f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:33.713301 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ed269-08f3-4a63-8031-51ee1ed636f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:33.713581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ed269-08f3-4a63-8031-51ee1ed636f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:33.714287 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ed269-08f3-4a63-8031-51ee1ed636f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:33.715652 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 909/1844] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:33.715940 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ed269-08f3-4a63-8031-51ee1ed636f0 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:33.733700 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a64b7e-8f18-4dab-82d8-7fb31b4bf509 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:33.737484 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 936/1845] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:33.738714 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ed269-08f3-4a63-8031-51ee1ed636f0 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:33.755281 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ed269-08f3-4a63-8031-51ee1ed636f0 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:33.756211 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a64b7e-8f18-4dab-82d8-7fb31b4bf509 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:33.761210 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979e7e7-8a15-4c6c-95a3-f1eeda0a5123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:33.761972 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979e7e7-8a15-4c6c-95a3-f1eeda0a5123 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:33.762339 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979e7e7-8a15-4c6c-95a3-f1eeda0a5123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:33.762749 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979e7e7-8a15-4c6c-95a3-f1eeda0a5123 admin admin] PATH_INFO: `/v3/projects/96b9402f84eb459d8ec2ae07e8d3ba2c/users/9fbc33a02da94b1a85a127b849c25949/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:33.773242 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d6b54-54dc-4668-a14f-75fa8e067f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:33.773955 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d6b54-54dc-4668-a14f-75fa8e067f52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:33.774248 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d6b54-54dc-4668-a14f-75fa8e067f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:33.774535 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d6b54-54dc-4668-a14f-75fa8e067f52 admin admin] PATH_INFO: `/v3/projects/70560ff732f0442494762af8e714fe7e/users/9a8fc4c4871e4511b8cb7556d6861463/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:33.780681 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1979e7e7-8a15-4c6c-95a3-f1eeda0a5123 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96b9402f84eb459d8ec2ae07e8d3ba2c, user_id=9fbc33a02da94b1a85a127b849c25949, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:33.784803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1979e7e7-8a15-4c6c-95a3-f1eeda0a5123 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:33.805862 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88d6b54-54dc-4668-a14f-75fa8e067f52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70560ff732f0442494762af8e714fe7e, user_id=9a8fc4c4871e4511b8cb7556d6861463, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:33.807132 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 937/1846] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:33 2022] PUT /identity/v3/projects/96b9402f84eb459d8ec2ae07e8d3ba2c/users/9fbc33a02da94b1a85a127b849c25949/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:33.808754 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88d6b54-54dc-4668-a14f-75fa8e067f52 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:33.810943 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979e7e7-8a15-4c6c-95a3-f1eeda0a5123 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:33.830361 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 910/1847] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:33 2022] PUT /identity/v3/projects/70560ff732f0442494762af8e714fe7e/users/9a8fc4c4871e4511b8cb7556d6861463/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:33.834379 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d6b54-54dc-4668-a14f-75fa8e067f52 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:33.883310 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:33.895331 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979e7e7-8a15-4c6c-95a3-f1eeda0a5123 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:33.900039 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb97981-dd40-4c21-93f4-077dc6551d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:33.900692 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb97981-dd40-4c21-93f4-077dc6551d3b admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:33.900982 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb97981-dd40-4c21-93f4-077dc6551d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:33.901080 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d6b54-54dc-4668-a14f-75fa8e067f52 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:33.901265 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb97981-dd40-4c21-93f4-077dc6551d3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:33.901972 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb97981-dd40-4c21-93f4-077dc6551d3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:33.903616 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb97981-dd40-4c21-93f4-077dc6551d3b admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:33.905983 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ff63f-086e-43bc-bee6-fe24664d5a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:33.906603 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ff63f-086e-43bc-bee6-fe24664d5a64 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:33.906954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ff63f-086e-43bc-bee6-fe24664d5a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:33.907241 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ff63f-086e-43bc-bee6-fe24664d5a64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:33.907946 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ff63f-086e-43bc-bee6-fe24664d5a64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:33.909586 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ff63f-086e-43bc-bee6-fe24664d5a64 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:33.913853 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 938/1848] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:33.918029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb97981-dd40-4c21-93f4-077dc6551d3b admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:33.922488 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 911/1849] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:33.926431 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ff63f-086e-43bc-bee6-fe24664d5a64 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:33.927679 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb97981-dd40-4c21-93f4-077dc6551d3b admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:33.932892 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8185ad-829a-4667-bdc7-5c8870ef7949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:33.933552 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8185ad-829a-4667-bdc7-5c8870ef7949 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:33.933839 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8185ad-829a-4667-bdc7-5c8870ef7949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:33.934119 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8185ad-829a-4667-bdc7-5c8870ef7949 admin admin] PATH_INFO: `/v3/projects/96b9402f84eb459d8ec2ae07e8d3ba2c/users/9fbc33a02da94b1a85a127b849c25949/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:33.936063 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8185ad-829a-4667-bdc7-5c8870ef7949 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96b9402f84eb459d8ec2ae07e8d3ba2c, user_id=9fbc33a02da94b1a85a127b849c25949, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:33.936704 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ff63f-086e-43bc-bee6-fe24664d5a64 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:33.938732 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8185ad-829a-4667-bdc7-5c8870ef7949 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:33.941402 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc64fb-4318-4681-91b7-40d0a5759b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:33.942057 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc64fb-4318-4681-91b7-40d0a5759b63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:33.942344 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc64fb-4318-4681-91b7-40d0a5759b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:33.942630 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc64fb-4318-4681-91b7-40d0a5759b63 admin admin] PATH_INFO: `/v3/projects/70560ff732f0442494762af8e714fe7e/users/9a8fc4c4871e4511b8cb7556d6861463/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:33.944635 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fc64fb-4318-4681-91b7-40d0a5759b63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70560ff732f0442494762af8e714fe7e, user_id=9a8fc4c4871e4511b8cb7556d6861463, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:33.947998 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fc64fb-4318-4681-91b7-40d0a5759b63 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:33.950935 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 939/1850] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:33 2022] PUT /identity/v3/projects/96b9402f84eb459d8ec2ae07e8d3ba2c/users/9fbc33a02da94b1a85a127b849c25949/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:33.962147 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedf3d1-8d54-41e6-a549-f27c85fd0cc4 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:33.962439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedf3d1-8d54-41e6-a549-f27c85fd0cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:33.962720 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedf3d1-8d54-41e6-a549-f27c85fd0cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:33.964354 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 912/1851] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:33 2022] PUT /identity/v3/projects/70560ff732f0442494762af8e714fe7e/users/9a8fc4c4871e4511b8cb7556d6861463/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:33.974867 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b57aff-e2b3-4656-96f9-40535f58b7d8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:33.975193 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b57aff-e2b3-4656-96f9-40535f58b7d8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:33.975564 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b57aff-e2b3-4656-96f9-40535f58b7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:34.022091 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-24b57aff-e2b3-4656-96f9-40535f58b7d8 None None] MFA Rules not processed for user `9a8fc4c4871e4511b8cb7556d6861463`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:34.023661 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-24b57aff-e2b3-4656-96f9-40535f58b7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:34.024163 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-7eedf3d1-8d54-41e6-a549-f27c85fd0cc4 None None] MFA Rules not processed for user `9fbc33a02da94b1a85a127b849c25949`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:34.025473 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-7eedf3d1-8d54-41e6-a549-f27c85fd0cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:34.097173 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:34.148605 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 913/1852] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:33 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:34.161726 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b57aff-e2b3-4656-96f9-40535f58b7d8 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:34.164256 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-24b57aff-e2b3-4656-96f9-40535f58b7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:34.213616 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 940/1853] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:33 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:34.227284 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eedf3d1-8d54-41e6-a549-f27c85fd0cc4 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:34.228934 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-7eedf3d1-8d54-41e6-a549-f27c85fd0cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:34.253160 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b57aff-e2b3-4656-96f9-40535f58b7d8 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:34.258951 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cef48f-6aa1-486f-ab39-860c4f749e5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:34.259641 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cef48f-6aa1-486f-ab39-860c4f749e5c service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:34.259935 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cef48f-6aa1-486f-ab39-860c4f749e5c service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:34.260223 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cef48f-6aa1-486f-ab39-860c4f749e5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:34.328594 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cef48f-6aa1-486f-ab39-860c4f749e5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:34.330140 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cef48f-6aa1-486f-ab39-860c4f749e5c service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:34.342552 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 914/1854] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:09:34.373189 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eedf3d1-8d54-41e6-a549-f27c85fd0cc4 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:34.378564 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d2af4-6ebe-435b-a245-4bbdf66fae91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:34.379435 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d2af4-6ebe-435b-a245-4bbdf66fae91 service cinder] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:34.379727 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d2af4-6ebe-435b-a245-4bbdf66fae91 service cinder] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:34.380009 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d2af4-6ebe-435b-a245-4bbdf66fae91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:34.392914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5d2af4-6ebe-435b-a245-4bbdf66fae91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:34.394656 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5d2af4-6ebe-435b-a245-4bbdf66fae91 service cinder] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:34.406237 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 941/1855] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:34 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:34.591875 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cef48f-6aa1-486f-ab39-860c4f749e5c service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:34.597481 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:34.601371 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cef48f-6aa1-486f-ab39-860c4f749e5c service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:34.615163 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea8ccc-7c35-4748-85f8-0eeda978f2c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:34.615804 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea8ccc-7c35-4748-85f8-0eeda978f2c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:34.616090 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea8ccc-7c35-4748-85f8-0eeda978f2c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:34.616373 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea8ccc-7c35-4748-85f8-0eeda978f2c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:34.749807 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ea8ccc-7c35-4748-85f8-0eeda978f2c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:34.751468 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ea8ccc-7c35-4748-85f8-0eeda978f2c8 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:34.768094 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 915/1856] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:34 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:40.364100 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d2af4-6ebe-435b-a245-4bbdf66fae91 service cinder] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:40.365597 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-da5d2af4-6ebe-435b-a245-4bbdf66fae91 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:40.445932 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d2af4-6ebe-435b-a245-4bbdf66fae91 service cinder] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:40.451911 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8352dfe-cedb-4afa-9083-471e08eeacd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:40.452112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8352dfe-cedb-4afa-9083-471e08eeacd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:40.452348 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8352dfe-cedb-4afa-9083-471e08eeacd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:40.452726 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8352dfe-cedb-4afa-9083-471e08eeacd1 admin admin] PATH_INFO: `/v3/users/3407935f4ae74dc7aa63466001a42805` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:40.454036 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8352dfe-cedb-4afa-9083-471e08eeacd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3407935f4ae74dc7aa63466001a42805)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:40.456754 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8352dfe-cedb-4afa-9083-471e08eeacd1 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:40.514927 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d8352dfe-cedb-4afa-9083-471e08eeacd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3407935f4ae74dc7aa63466001a42805'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:40.516009 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d8352dfe-cedb-4afa-9083-471e08eeacd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3407935f4ae74dc7aa63466001a42805'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:40.536236 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d8352dfe-cedb-4afa-9083-471e08eeacd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3407935f4ae74dc7aa63466001a42805'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:40.555340 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d8352dfe-cedb-4afa-9083-471e08eeacd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3407935f4ae74dc7aa63466001a42805'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:40.556660 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d8352dfe-cedb-4afa-9083-471e08eeacd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3407935f4ae74dc7aa63466001a42805'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:40.566818 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:40.604803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 942/1857] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:40 2022] DELETE /identity/v3/users/3407935f4ae74dc7aa63466001a42805 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:40.609501 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa61bd-65c6-47c7-b136-a113b1514543 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:40.609961 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa61bd-65c6-47c7-b136-a113b1514543 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:40.610171 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa61bd-65c6-47c7-b136-a113b1514543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:40.657249 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4caa61bd-65c6-47c7-b136-a113b1514543 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:40.658827 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4caa61bd-65c6-47c7-b136-a113b1514543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:40.736618 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 916/1858] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:40.746547 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8352dfe-cedb-4afa-9083-471e08eeacd1 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:40.748075 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-d8352dfe-cedb-4afa-9083-471e08eeacd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:40.834062 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8352dfe-cedb-4afa-9083-471e08eeacd1 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:40.844580 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb42dc-6755-47f1-82a8-8357fa814091 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:40.845249 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb42dc-6755-47f1-82a8-8357fa814091 service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:40.845536 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb42dc-6755-47f1-82a8-8357fa814091 service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:40.845939 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb42dc-6755-47f1-82a8-8357fa814091 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:40.857923 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb42dc-6755-47f1-82a8-8357fa814091 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:40.859557 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb42dc-6755-47f1-82a8-8357fa814091 service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:40.883259 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 943/1859] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:41.239750 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa61bd-65c6-47c7-b136-a113b1514543 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:41.242796 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4caa61bd-65c6-47c7-b136-a113b1514543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:41.272853 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa61bd-65c6-47c7-b136-a113b1514543 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:41.282371 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e692cea-f0e9-4713-8969-2c5a6d878de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:41.283291 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e692cea-f0e9-4713-8969-2c5a6d878de7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:41.283625 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e692cea-f0e9-4713-8969-2c5a6d878de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:41.284129 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e692cea-f0e9-4713-8969-2c5a6d878de7 admin admin] PATH_INFO: `/v3/projects/21e82617f6e64fafae1b98fa167be7ed` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:41.285611 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e692cea-f0e9-4713-8969-2c5a6d878de7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21e82617f6e64fafae1b98fa167be7ed)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:41.287819 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e692cea-f0e9-4713-8969-2c5a6d878de7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:41.348137 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9e692cea-f0e9-4713-8969-2c5a6d878de7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21e82617f6e64fafae1b98fa167be7ed', 'request_id': 'req-9e692cea-f0e9-4713-8969-2c5a6d878de7'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:41.356863 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9e692cea-f0e9-4713-8969-2c5a6d878de7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21e82617f6e64fafae1b98fa167be7ed', 'request_id': 'req-9e692cea-f0e9-4713-8969-2c5a6d878de7'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:41.365146 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9e692cea-f0e9-4713-8969-2c5a6d878de7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21e82617f6e64fafae1b98fa167be7ed', 'request_id': 'req-9e692cea-f0e9-4713-8969-2c5a6d878de7'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:41.397173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9e692cea-f0e9-4713-8969-2c5a6d878de7 admin admin] The token cache is being invalidate because project 21e82617f6e64fafae1b98fa167be7ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:41.397510 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9e692cea-f0e9-4713-8969-2c5a6d878de7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:41.398978 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9e692cea-f0e9-4713-8969-2c5a6d878de7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:41.400953 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 917/1860] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:41 2022] DELETE /identity/v3/projects/21e82617f6e64fafae1b98fa167be7ed => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:41.405533 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb42dc-6755-47f1-82a8-8357fa814091 service neutron] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:41.417021 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb42dc-6755-47f1-82a8-8357fa814091 service neutron] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:41.492893 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118b6d7-b2a3-4b83-9e0d-0e9ecc1bfa97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:41.494089 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118b6d7-b2a3-4b83-9e0d-0e9ecc1bfa97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:41.494089 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118b6d7-b2a3-4b83-9e0d-0e9ecc1bfa97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:41.494089 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118b6d7-b2a3-4b83-9e0d-0e9ecc1bfa97 admin admin] PATH_INFO: `/v3/users/f435018dcd4f4e1dab8b9abc822caf1f` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:41.495155 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4118b6d7-b2a3-4b83-9e0d-0e9ecc1bfa97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f435018dcd4f4e1dab8b9abc822caf1f)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:41.500363 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4118b6d7-b2a3-4b83-9e0d-0e9ecc1bfa97 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:41.570207 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-4118b6d7-b2a3-4b83-9e0d-0e9ecc1bfa97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f435018dcd4f4e1dab8b9abc822caf1f'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:41.570943 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-4118b6d7-b2a3-4b83-9e0d-0e9ecc1bfa97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f435018dcd4f4e1dab8b9abc822caf1f'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:41.586329 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-4118b6d7-b2a3-4b83-9e0d-0e9ecc1bfa97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f435018dcd4f4e1dab8b9abc822caf1f'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:41.595462 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-4118b6d7-b2a3-4b83-9e0d-0e9ecc1bfa97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f435018dcd4f4e1dab8b9abc822caf1f'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:41.596203 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-4118b6d7-b2a3-4b83-9e0d-0e9ecc1bfa97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f435018dcd4f4e1dab8b9abc822caf1f'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:41.609368 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 944/1861] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:41 2022] DELETE /identity/v3/users/f435018dcd4f4e1dab8b9abc822caf1f => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:42.025394 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e692cea-f0e9-4713-8969-2c5a6d878de7 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:42.026984 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9e692cea-f0e9-4713-8969-2c5a6d878de7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:42.044572 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e692cea-f0e9-4713-8969-2c5a6d878de7 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:42.073473 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:42.113295 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d7f4d-76e9-40b1-9a77-6c4e6f945bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:42.113954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d7f4d-76e9-40b1-9a77-6c4e6f945bdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:42.114313 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d7f4d-76e9-40b1-9a77-6c4e6f945bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:42.114603 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d7f4d-76e9-40b1-9a77-6c4e6f945bdd admin admin] PATH_INFO: `/v3/projects/ec59270f6d3044fd80d708e44c6b7bd4` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:42.117406 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5d7f4d-76e9-40b1-9a77-6c4e6f945bdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec59270f6d3044fd80d708e44c6b7bd4)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:42.119538 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5d7f4d-76e9-40b1-9a77-6c4e6f945bdd admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:42.171097 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118b6d7-b2a3-4b83-9e0d-0e9ecc1bfa97 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:42.173410 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-4118b6d7-b2a3-4b83-9e0d-0e9ecc1bfa97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:42.184565 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6e5d7f4d-76e9-40b1-9a77-6c4e6f945bdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec59270f6d3044fd80d708e44c6b7bd4', 'request_id': 'req-6e5d7f4d-76e9-40b1-9a77-6c4e6f945bdd'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:42.194971 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6e5d7f4d-76e9-40b1-9a77-6c4e6f945bdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec59270f6d3044fd80d708e44c6b7bd4', 'request_id': 'req-6e5d7f4d-76e9-40b1-9a77-6c4e6f945bdd'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:42.199282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:42.204765 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6e5d7f4d-76e9-40b1-9a77-6c4e6f945bdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec59270f6d3044fd80d708e44c6b7bd4', 'request_id': 'req-6e5d7f4d-76e9-40b1-9a77-6c4e6f945bdd'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:42.220839 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6e5d7f4d-76e9-40b1-9a77-6c4e6f945bdd admin admin] The token cache is being invalidate because project ec59270f6d3044fd80d708e44c6b7bd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:42.221135 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6e5d7f4d-76e9-40b1-9a77-6c4e6f945bdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:42.222032 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6e5d7f4d-76e9-40b1-9a77-6c4e6f945bdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:42.224807 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 918/1862] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:42 2022] DELETE /identity/v3/projects/ec59270f6d3044fd80d708e44c6b7bd4 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:42.229889 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d7f4d-76e9-40b1-9a77-6c4e6f945bdd admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:42.231278 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6e5d7f4d-76e9-40b1-9a77-6c4e6f945bdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:42.253481 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118b6d7-b2a3-4b83-9e0d-0e9ecc1bfa97 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:42.315717 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6373b687-a92d-494c-8d28-91c5b97ad4cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:42.316449 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373b687-a92d-494c-8d28-91c5b97ad4cb service cinder] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:42.316819 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373b687-a92d-494c-8d28-91c5b97ad4cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:42.317169 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373b687-a92d-494c-8d28-91c5b97ad4cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:42.319063 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-6373b687-a92d-494c-8d28-91c5b97ad4cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:42.319737 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d7f4d-76e9-40b1-9a77-6c4e6f945bdd admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:42.324575 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e2df3b-459d-40ee-b8f1-47209b54b8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:42.325294 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e2df3b-459d-40ee-b8f1-47209b54b8da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:42.325579 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e2df3b-459d-40ee-b8f1-47209b54b8da admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:42.325856 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e2df3b-459d-40ee-b8f1-47209b54b8da admin admin] PATH_INFO: `/v3/users/a8e7c83862b44dfd89a1c21fafbf4caa` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:42.327352 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e2df3b-459d-40ee-b8f1-47209b54b8da admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8e7c83862b44dfd89a1c21fafbf4caa)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:42.329368 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e2df3b-459d-40ee-b8f1-47209b54b8da admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:42.405871 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-77e2df3b-459d-40ee-b8f1-47209b54b8da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8e7c83862b44dfd89a1c21fafbf4caa'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:42.411645 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-77e2df3b-459d-40ee-b8f1-47209b54b8da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8e7c83862b44dfd89a1c21fafbf4caa'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:42.412331 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-77e2df3b-459d-40ee-b8f1-47209b54b8da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8e7c83862b44dfd89a1c21fafbf4caa'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:42.415072 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:42.419177 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6373b687-a92d-494c-8d28-91c5b97ad4cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:42.420755 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6373b687-a92d-494c-8d28-91c5b97ad4cb service cinder] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:42.420949 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-77e2df3b-459d-40ee-b8f1-47209b54b8da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8e7c83862b44dfd89a1c21fafbf4caa'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:42.421590 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-77e2df3b-459d-40ee-b8f1-47209b54b8da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8e7c83862b44dfd89a1c21fafbf4caa'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:42.431202 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 945/1863] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:42 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:42.450031 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 919/1864] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:42 2022] DELETE /identity/v3/users/a8e7c83862b44dfd89a1c21fafbf4caa => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:42.921242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6373b687-a92d-494c-8d28-91c5b97ad4cb service cinder] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:42.938211 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e2df3b-459d-40ee-b8f1-47209b54b8da admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:42.941212 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-77e2df3b-459d-40ee-b8f1-47209b54b8da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:42.948828 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6373b687-a92d-494c-8d28-91c5b97ad4cb service cinder] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:43.075254 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a548420d-4b5e-4548-89a7-5783399f598f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:43.076322 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548420d-4b5e-4548-89a7-5783399f598f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:43.076879 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548420d-4b5e-4548-89a7-5783399f598f admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:43.077368 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548420d-4b5e-4548-89a7-5783399f598f admin admin] PATH_INFO: `/v3/projects/2593a76a19dc4fbd9eb1f09a21d400b6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:43.078783 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a548420d-4b5e-4548-89a7-5783399f598f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2593a76a19dc4fbd9eb1f09a21d400b6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:43.081384 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a548420d-4b5e-4548-89a7-5783399f598f admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:43.100209 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e2df3b-459d-40ee-b8f1-47209b54b8da admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:43.113897 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:43.123829 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1a9b2-1c37-4e42-ade3-837617e202a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:43.124677 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1a9b2-1c37-4e42-ade3-837617e202a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:43.124994 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1a9b2-1c37-4e42-ade3-837617e202a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:43.125269 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1a9b2-1c37-4e42-ade3-837617e202a8 admin admin] PATH_INFO: `/v3/users/37238a9211c347c48a2b4a4f2eff69c4` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:43.126500 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f1a9b2-1c37-4e42-ade3-837617e202a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37238a9211c347c48a2b4a4f2eff69c4)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:43.128464 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f1a9b2-1c37-4e42-ade3-837617e202a8 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:43.150781 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-a548420d-4b5e-4548-89a7-5783399f598f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2593a76a19dc4fbd9eb1f09a21d400b6', 'request_id': 'req-a548420d-4b5e-4548-89a7-5783399f598f'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:43.159895 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-a548420d-4b5e-4548-89a7-5783399f598f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2593a76a19dc4fbd9eb1f09a21d400b6', 'request_id': 'req-a548420d-4b5e-4548-89a7-5783399f598f'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:43.168109 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-a548420d-4b5e-4548-89a7-5783399f598f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2593a76a19dc4fbd9eb1f09a21d400b6', 'request_id': 'req-a548420d-4b5e-4548-89a7-5783399f598f'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:43.186873 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-a548420d-4b5e-4548-89a7-5783399f598f admin admin] The token cache is being invalidate because project 2593a76a19dc4fbd9eb1f09a21d400b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:43.187448 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-a548420d-4b5e-4548-89a7-5783399f598f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:43.188471 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-a548420d-4b5e-4548-89a7-5783399f598f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:43.190666 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 946/1865] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:42 2022] DELETE /identity/v3/projects/2593a76a19dc4fbd9eb1f09a21d400b6 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:43.200346 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb0b36-7914-4364-8006-e3958d2ae872 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:43.200669 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb0b36-7914-4364-8006-e3958d2ae872 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:43.201053 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb0b36-7914-4364-8006-e3958d2ae872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:43.222359 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a9f1a9b2-1c37-4e42-ade3-837617e202a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37238a9211c347c48a2b4a4f2eff69c4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:43.233307 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-bdeb0b36-7914-4364-8006-e3958d2ae872 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:43.234244 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a9f1a9b2-1c37-4e42-ade3-837617e202a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37238a9211c347c48a2b4a4f2eff69c4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:43.234602 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-bdeb0b36-7914-4364-8006-e3958d2ae872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:43.235843 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a9f1a9b2-1c37-4e42-ade3-837617e202a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37238a9211c347c48a2b4a4f2eff69c4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:43.249409 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a9f1a9b2-1c37-4e42-ade3-837617e202a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37238a9211c347c48a2b4a4f2eff69c4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:43.250433 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a9f1a9b2-1c37-4e42-ade3-837617e202a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37238a9211c347c48a2b4a4f2eff69c4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:43.279542 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 920/1866] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:42 2022] DELETE /identity/v3/users/37238a9211c347c48a2b4a4f2eff69c4 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:43.282172 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7eda31-315e-49ec-a129-9803a40474d2 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:43.282460 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7eda31-315e-49ec-a129-9803a40474d2 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:43.282732 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7eda31-315e-49ec-a129-9803a40474d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:43.301862 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 947/1867] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:43.302031 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-2b7eda31-315e-49ec-a129-9803a40474d2 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:43.303189 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2b7eda31-315e-49ec-a129-9803a40474d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:43.311859 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b86916-78a2-44b8-a659-0fca122d225b None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:43.312151 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b86916-78a2-44b8-a659-0fca122d225b None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:43.312434 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b86916-78a2-44b8-a659-0fca122d225b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:43.331471 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-f1b86916-78a2-44b8-a659-0fca122d225b None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:43.333009 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-f1b86916-78a2-44b8-a659-0fca122d225b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:43.351646 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:43.365701 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 921/1868] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:43.375664 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7eda31-315e-49ec-a129-9803a40474d2 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:43.377049 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2b7eda31-315e-49ec-a129-9803a40474d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:43.398576 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 948/1869] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:43.402697 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b86916-78a2-44b8-a659-0fca122d225b None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:43.412168 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b86916-78a2-44b8-a659-0fca122d225b None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:43.418131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1978b003-2450-4cac-a0c5-d4558ff0534c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:43.418755 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978b003-2450-4cac-a0c5-d4558ff0534c admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:43.419038 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978b003-2450-4cac-a0c5-d4558ff0534c admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:43.419315 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978b003-2450-4cac-a0c5-d4558ff0534c admin admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:43.420021 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1978b003-2450-4cac-a0c5-d4558ff0534c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:43.420326 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1978b003-2450-4cac-a0c5-d4558ff0534c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:43.422098 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1978b003-2450-4cac-a0c5-d4558ff0534c admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:43.434299 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 949/1870] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:09:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:09:43.438916 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1978b003-2450-4cac-a0c5-d4558ff0534c admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:43.445059 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7eda31-315e-49ec-a129-9803a40474d2 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:43.449063 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c2306-e9a9-4e58-9433-a57aaa2571ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:43.449790 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1978b003-2450-4cac-a0c5-d4558ff0534c admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:43.449947 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c2306-e9a9-4e58-9433-a57aaa2571ff service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:43.450222 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c2306-e9a9-4e58-9433-a57aaa2571ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:43.450502 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c2306-e9a9-4e58-9433-a57aaa2571ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:43.454717 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3fa66c-8f34-4c51-bd00-8c5537405c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:43.455335 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3fa66c-8f34-4c51-bd00-8c5537405c3e admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:43.455622 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3fa66c-8f34-4c51-bd00-8c5537405c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:43.455905 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3fa66c-8f34-4c51-bd00-8c5537405c3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:43.456634 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3fa66c-8f34-4c51-bd00-8c5537405c3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:43.458571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3fa66c-8f34-4c51-bd00-8c5537405c3e admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:43.461905 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67c2306-e9a9-4e58-9433-a57aaa2571ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:43.463543 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67c2306-e9a9-4e58-9433-a57aaa2571ff service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:43.474405 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 922/1871] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:09:43.490684 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 950/1872] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:43 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:09:43.495431 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c2306-e9a9-4e58-9433-a57aaa2571ff service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:43.507441 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c2306-e9a9-4e58-9433-a57aaa2571ff service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:43.569951 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c038d56-086d-42c6-9b8f-4334f1bbfcce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:43.570652 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c038d56-086d-42c6-9b8f-4334f1bbfcce admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:43.570995 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c038d56-086d-42c6-9b8f-4334f1bbfcce admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:43.571332 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c038d56-086d-42c6-9b8f-4334f1bbfcce admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:43.572171 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c038d56-086d-42c6-9b8f-4334f1bbfcce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:43.575828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c038d56-086d-42c6-9b8f-4334f1bbfcce admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:43.603016 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 923/1873] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:43 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:43.604259 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3fa66c-8f34-4c51-bd00-8c5537405c3e admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:43.615759 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3fa66c-8f34-4c51-bd00-8c5537405c3e admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:43.621545 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e1eda-db5f-4438-b59b-91a520acfacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:43.622182 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e1eda-db5f-4438-b59b-91a520acfacc admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:43.622468 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e1eda-db5f-4438-b59b-91a520acfacc admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:43.622747 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e1eda-db5f-4438-b59b-91a520acfacc admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:43.623489 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e1eda-db5f-4438-b59b-91a520acfacc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:43.625305 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e1eda-db5f-4438-b59b-91a520acfacc admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:43.637800 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 951/1874] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:43.641979 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c038d56-086d-42c6-9b8f-4334f1bbfcce admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:43.652283 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c038d56-086d-42c6-9b8f-4334f1bbfcce admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:43.657132 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166e0eb9-7e5a-4440-9fdd-7f887976a2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:43.657759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166e0eb9-7e5a-4440-9fdd-7f887976a2b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:43.658041 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166e0eb9-7e5a-4440-9fdd-7f887976a2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:43.658314 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166e0eb9-7e5a-4440-9fdd-7f887976a2b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:43.659005 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166e0eb9-7e5a-4440-9fdd-7f887976a2b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:43.660622 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166e0eb9-7e5a-4440-9fdd-7f887976a2b7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:43.672341 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 924/1875] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:43.677442 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e1eda-db5f-4438-b59b-91a520acfacc admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:43.695369 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e1eda-db5f-4438-b59b-91a520acfacc admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:43.701061 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb16a7a-9a4c-4a78-8cb8-073406151bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:43.701771 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb16a7a-9a4c-4a78-8cb8-073406151bf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:43.702059 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb16a7a-9a4c-4a78-8cb8-073406151bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:43.702343 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb16a7a-9a4c-4a78-8cb8-073406151bf2 admin admin] PATH_INFO: `/v3/projects/5324c3957e004c548e5fb243d6a23aaf/users/4e5b738a1ef6499cb2c9447c80b007a2/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:43.705464 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:43.728580 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb16a7a-9a4c-4a78-8cb8-073406151bf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5324c3957e004c548e5fb243d6a23aaf, user_id=4e5b738a1ef6499cb2c9447c80b007a2, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:43.732637 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb16a7a-9a4c-4a78-8cb8-073406151bf2 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:43.756475 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 952/1876] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:43 2022] PUT /identity/v3/projects/5324c3957e004c548e5fb243d6a23aaf/users/4e5b738a1ef6499cb2c9447c80b007a2/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:43.769632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837e52c-0bf6-4ef4-b17b-fcf59d19c38b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:43.769986 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837e52c-0bf6-4ef4-b17b-fcf59d19c38b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:43.770345 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837e52c-0bf6-4ef4-b17b-fcf59d19c38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:43.806458 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-1837e52c-0bf6-4ef4-b17b-fcf59d19c38b None None] MFA Rules not processed for user `4e5b738a1ef6499cb2c9447c80b007a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:43.808251 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1837e52c-0bf6-4ef4-b17b-fcf59d19c38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:43.843711 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb16a7a-9a4c-4a78-8cb8-073406151bf2 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:43.845676 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-abb16a7a-9a4c-4a78-8cb8-073406151bf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:43.905406 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 925/1877] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:43.906577 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1837e52c-0bf6-4ef4-b17b-fcf59d19c38b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:43.908100 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1837e52c-0bf6-4ef4-b17b-fcf59d19c38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:43.920588 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:43.962213 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb16a7a-9a4c-4a78-8cb8-073406151bf2 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:43.969617 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd43b1fa-09c8-4e60-aa4e-495ac1c4f05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:43.970241 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd43b1fa-09c8-4e60-aa4e-495ac1c4f05f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:43.970600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd43b1fa-09c8-4e60-aa4e-495ac1c4f05f admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:43.970914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd43b1fa-09c8-4e60-aa4e-495ac1c4f05f admin admin] PATH_INFO: `/v3/users/047580fe85ef493d9de6c8831cb2d605` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:43.972422 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd43b1fa-09c8-4e60-aa4e-495ac1c4f05f admin admin] RBAC: Authorizing `identity:delete_user(user_id=047580fe85ef493d9de6c8831cb2d605)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:43.974748 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd43b1fa-09c8-4e60-aa4e-495ac1c4f05f admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:44.017576 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1837e52c-0bf6-4ef4-b17b-fcf59d19c38b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:44.025569 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb383977-b5e6-4ab3-b0c8-209dba675a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:44.026301 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb383977-b5e6-4ab3-b0c8-209dba675a85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:44.026595 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb383977-b5e6-4ab3-b0c8-209dba675a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:44.026879 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb383977-b5e6-4ab3-b0c8-209dba675a85 admin admin] PATH_INFO: `/v3/projects/77b5b204348840febd02752d62ca153e` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:44.028457 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb383977-b5e6-4ab3-b0c8-209dba675a85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77b5b204348840febd02752d62ca153e)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:44.030984 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb383977-b5e6-4ab3-b0c8-209dba675a85 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:44.041506 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-dd43b1fa-09c8-4e60-aa4e-495ac1c4f05f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '047580fe85ef493d9de6c8831cb2d605'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:44.050740 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-dd43b1fa-09c8-4e60-aa4e-495ac1c4f05f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '047580fe85ef493d9de6c8831cb2d605'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:44.071120 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-dd43b1fa-09c8-4e60-aa4e-495ac1c4f05f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '047580fe85ef493d9de6c8831cb2d605'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:44.088383 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-dd43b1fa-09c8-4e60-aa4e-495ac1c4f05f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '047580fe85ef493d9de6c8831cb2d605'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:44.089498 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-dd43b1fa-09c8-4e60-aa4e-495ac1c4f05f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '047580fe85ef493d9de6c8831cb2d605'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:44.105824 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-fb383977-b5e6-4ab3-b0c8-209dba675a85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77b5b204348840febd02752d62ca153e', 'request_id': 'req-fb383977-b5e6-4ab3-b0c8-209dba675a85'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:44.110808 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 953/1878] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:43 2022] DELETE /identity/v3/users/047580fe85ef493d9de6c8831cb2d605 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:44.112052 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd43b1fa-09c8-4e60-aa4e-495ac1c4f05f admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:44.115377 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-dd43b1fa-09c8-4e60-aa4e-495ac1c4f05f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:44.118170 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-fb383977-b5e6-4ab3-b0c8-209dba675a85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77b5b204348840febd02752d62ca153e', 'request_id': 'req-fb383977-b5e6-4ab3-b0c8-209dba675a85'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:44.135701 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-fb383977-b5e6-4ab3-b0c8-209dba675a85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77b5b204348840febd02752d62ca153e', 'request_id': 'req-fb383977-b5e6-4ab3-b0c8-209dba675a85'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:44.151240 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-fb383977-b5e6-4ab3-b0c8-209dba675a85 admin admin] The token cache is being invalidate because project 77b5b204348840febd02752d62ca153e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:44.151613 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-fb383977-b5e6-4ab3-b0c8-209dba675a85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:44.152441 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-fb383977-b5e6-4ab3-b0c8-209dba675a85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:44.154398 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 926/1879] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:43 2022] DELETE /identity/v3/projects/77b5b204348840febd02752d62ca153e => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:44.158626 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ad442-59fa-4483-82c6-d26ce82c07ce None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:44.159059 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ad442-59fa-4483-82c6-d26ce82c07ce None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:44.159468 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ad442-59fa-4483-82c6-d26ce82c07ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:44.186441 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-530ad442-59fa-4483-82c6-d26ce82c07ce None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:44.187729 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-530ad442-59fa-4483-82c6-d26ce82c07ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:44.261822 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:44.277974 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd43b1fa-09c8-4e60-aa4e-495ac1c4f05f admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:44.284623 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58573005-4a0c-44f8-b926-2738d994c7da service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39a66f7362964389be039937a786bb0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:44.284623 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58573005-4a0c-44f8-b926-2738d994c7da service glance] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:44.285673 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58573005-4a0c-44f8-b926-2738d994c7da service glance] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:44.285965 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58573005-4a0c-44f8-b926-2738d994c7da service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:44.288902 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-58573005-4a0c-44f8-b926-2738d994c7da service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:44.326009 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 927/1880] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:44.335963 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ad442-59fa-4483-82c6-d26ce82c07ce None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:44.344758 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-530ad442-59fa-4483-82c6-d26ce82c07ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:44.368225 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ad442-59fa-4483-82c6-d26ce82c07ce None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:44.374537 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a3557-f700-4282-9bc8-9590dbc2e196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:44.375261 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a3557-f700-4282-9bc8-9590dbc2e196 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:44.375553 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a3557-f700-4282-9bc8-9590dbc2e196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:44.375836 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a3557-f700-4282-9bc8-9590dbc2e196 admin admin] PATH_INFO: `/v3/users/9fbc33a02da94b1a85a127b849c25949` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:44.377248 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a3557-f700-4282-9bc8-9590dbc2e196 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fbc33a02da94b1a85a127b849c25949)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:44.379236 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a3557-f700-4282-9bc8-9590dbc2e196 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:44.394041 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58573005-4a0c-44f8-b926-2738d994c7da service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:44.396626 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58573005-4a0c-44f8-b926-2738d994c7da service glance] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:44.414479 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 954/1881] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:44.416331 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58573005-4a0c-44f8-b926-2738d994c7da service glance] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:44.418319 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-58573005-4a0c-44f8-b926-2738d994c7da service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:44.446009 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:44.469413 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1d9a3557-f700-4282-9bc8-9590dbc2e196 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fbc33a02da94b1a85a127b849c25949'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:44.480907 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1d9a3557-f700-4282-9bc8-9590dbc2e196 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fbc33a02da94b1a85a127b849c25949'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:44.482666 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1d9a3557-f700-4282-9bc8-9590dbc2e196 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fbc33a02da94b1a85a127b849c25949'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:44.494776 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1d9a3557-f700-4282-9bc8-9590dbc2e196 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fbc33a02da94b1a85a127b849c25949'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:44.496327 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1d9a3557-f700-4282-9bc8-9590dbc2e196 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fbc33a02da94b1a85a127b849c25949'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:44.509908 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58573005-4a0c-44f8-b926-2738d994c7da service glance] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:44.516052 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c53a3-fe2f-4394-ac6f-a06c999b01ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:44.517137 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c53a3-fe2f-4394-ac6f-a06c999b01ae service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:44.518059 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c53a3-fe2f-4394-ac6f-a06c999b01ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:44.518357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c53a3-fe2f-4394-ac6f-a06c999b01ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:44.520619 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-a08c53a3-fe2f-4394-ac6f-a06c999b01ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:44.525225 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 928/1882] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:44 2022] DELETE /identity/v3/users/9fbc33a02da94b1a85a127b849c25949 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:44.544767 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08c53a3-fe2f-4394-ac6f-a06c999b01ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:44.546339 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08c53a3-fe2f-4394-ac6f-a06c999b01ae service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:44.559208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 955/1883] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:45.243563 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a3557-f700-4282-9bc8-9590dbc2e196 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:45.245917 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1d9a3557-f700-4282-9bc8-9590dbc2e196 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:45.356043 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c53a3-fe2f-4394-ac6f-a06c999b01ae service neutron] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:45.358282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-a08c53a3-fe2f-4394-ac6f-a06c999b01ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:45.403811 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c53a3-fe2f-4394-ac6f-a06c999b01ae service neutron] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:45.422901 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a3557-f700-4282-9bc8-9590dbc2e196 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:45.429363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b140ce3-d532-4440-b587-81b6363c5162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:45.430040 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b140ce3-d532-4440-b587-81b6363c5162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:45.430328 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b140ce3-d532-4440-b587-81b6363c5162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:45.430613 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b140ce3-d532-4440-b587-81b6363c5162 admin admin] PATH_INFO: `/v3/projects/96b9402f84eb459d8ec2ae07e8d3ba2c` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:45.431890 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b140ce3-d532-4440-b587-81b6363c5162 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96b9402f84eb459d8ec2ae07e8d3ba2c)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:45.434176 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b140ce3-d532-4440-b587-81b6363c5162 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:45.506076 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0b140ce3-d532-4440-b587-81b6363c5162 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96b9402f84eb459d8ec2ae07e8d3ba2c', 'request_id': 'req-0b140ce3-d532-4440-b587-81b6363c5162'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:45.520415 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0b140ce3-d532-4440-b587-81b6363c5162 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96b9402f84eb459d8ec2ae07e8d3ba2c', 'request_id': 'req-0b140ce3-d532-4440-b587-81b6363c5162'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:45.542923 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0b140ce3-d532-4440-b587-81b6363c5162 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96b9402f84eb459d8ec2ae07e8d3ba2c', 'request_id': 'req-0b140ce3-d532-4440-b587-81b6363c5162'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:45.560694 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0b140ce3-d532-4440-b587-81b6363c5162 admin admin] The token cache is being invalidate because project 96b9402f84eb459d8ec2ae07e8d3ba2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:45.561037 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0b140ce3-d532-4440-b587-81b6363c5162 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:45.561668 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0b140ce3-d532-4440-b587-81b6363c5162 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:45.563213 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 929/1884] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:45 2022] DELETE /identity/v3/projects/96b9402f84eb459d8ec2ae07e8d3ba2c => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:45.568988 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8284d-073a-40aa-9a14-038ffda3f196 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:45.569650 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8284d-073a-40aa-9a14-038ffda3f196 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:45.570025 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8284d-073a-40aa-9a14-038ffda3f196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:45.582403 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894328f9-0dc8-4007-a5e0-191d89b87dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:45.583946 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894328f9-0dc8-4007-a5e0-191d89b87dee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:45.584665 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894328f9-0dc8-4007-a5e0-191d89b87dee admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:45.585275 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894328f9-0dc8-4007-a5e0-191d89b87dee admin admin] PATH_INFO: `/v3/projects/97ea1f5a5763441e985e38adfc4c81fb` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:45.587643 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894328f9-0dc8-4007-a5e0-191d89b87dee admin admin] RBAC: Authorizing `identity:delete_project(project_id=97ea1f5a5763441e985e38adfc4c81fb)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:45.590119 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894328f9-0dc8-4007-a5e0-191d89b87dee admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:45.641740 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-6bf8284d-073a-40aa-9a14-038ffda3f196 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:45.645298 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6bf8284d-073a-40aa-9a14-038ffda3f196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:45.700616 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:45.722471 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-894328f9-0dc8-4007-a5e0-191d89b87dee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97ea1f5a5763441e985e38adfc4c81fb', 'request_id': 'req-894328f9-0dc8-4007-a5e0-191d89b87dee'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:45.736704 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:45.750595 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-894328f9-0dc8-4007-a5e0-191d89b87dee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97ea1f5a5763441e985e38adfc4c81fb', 'request_id': 'req-894328f9-0dc8-4007-a5e0-191d89b87dee'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:45.761294 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-894328f9-0dc8-4007-a5e0-191d89b87dee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97ea1f5a5763441e985e38adfc4c81fb', 'request_id': 'req-894328f9-0dc8-4007-a5e0-191d89b87dee'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:45.785311 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-894328f9-0dc8-4007-a5e0-191d89b87dee admin admin] The token cache is being invalidate because project 97ea1f5a5763441e985e38adfc4c81fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:45.785632 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-894328f9-0dc8-4007-a5e0-191d89b87dee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:45.786374 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-894328f9-0dc8-4007-a5e0-191d89b87dee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:45.789900 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 956/1885] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:45 2022] DELETE /identity/v3/projects/97ea1f5a5763441e985e38adfc4c81fb => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:45.791593 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295cdc2-4a9d-4b33-8ef0-1bb139b4c8b9 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:45.792106 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295cdc2-4a9d-4b33-8ef0-1bb139b4c8b9 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:45.792512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295cdc2-4a9d-4b33-8ef0-1bb139b4c8b9 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:45.795136 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 957/1886] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 14:09:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:09:45.795788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9295cdc2-4a9d-4b33-8ef0-1bb139b4c8b9 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:45.800637 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-9295cdc2-4a9d-4b33-8ef0-1bb139b4c8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:45.813004 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 930/1887] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:45.815026 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58027c4-eba8-4134-bc78-6d7b8122b3ac None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:45.815983 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58027c4-eba8-4134-bc78-6d7b8122b3ac None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:45.816369 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58027c4-eba8-4134-bc78-6d7b8122b3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:45.854519 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d58027c4-eba8-4134-bc78-6d7b8122b3ac None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:45.856073 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d58027c4-eba8-4134-bc78-6d7b8122b3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:45.892865 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9295cdc2-4a9d-4b33-8ef0-1bb139b4c8b9 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:45.899906 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f6927-aafe-422a-bcb7-c103fff4a6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:45.903801 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f6927-aafe-422a-bcb7-c103fff4a6c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:45.904133 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f6927-aafe-422a-bcb7-c103fff4a6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:45.904442 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f6927-aafe-422a-bcb7-c103fff4a6c3 admin admin] PATH_INFO: `/v3/users/7a49246b355d43c4ba5c241e19bde45e` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:45.906246 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94f6927-aafe-422a-bcb7-c103fff4a6c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a49246b355d43c4ba5c241e19bde45e)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:45.908056 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94f6927-aafe-422a-bcb7-c103fff4a6c3 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:45.924462 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 931/1888] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:45.926121 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9e13a-89ab-4bd2-82c7-cf5f2613a0c6 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:45.926429 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9e13a-89ab-4bd2-82c7-cf5f2613a0c6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:45.926766 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9e13a-89ab-4bd2-82c7-cf5f2613a0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:45.949917 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-0cf9e13a-89ab-4bd2-82c7-cf5f2613a0c6 None None] MFA Rules not processed for user `a246654c783b4cc68d212e91bb05d8c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:45.951820 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0cf9e13a-89ab-4bd2-82c7-cf5f2613a0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:45.959320 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-c94f6927-aafe-422a-bcb7-c103fff4a6c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a49246b355d43c4ba5c241e19bde45e'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:45.960606 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-c94f6927-aafe-422a-bcb7-c103fff4a6c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a49246b355d43c4ba5c241e19bde45e'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:45.976315 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-c94f6927-aafe-422a-bcb7-c103fff4a6c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a49246b355d43c4ba5c241e19bde45e'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:45.987351 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-c94f6927-aafe-422a-bcb7-c103fff4a6c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a49246b355d43c4ba5c241e19bde45e'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:45.988120 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-c94f6927-aafe-422a-bcb7-c103fff4a6c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a49246b355d43c4ba5c241e19bde45e'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:46.034125 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 958/1889] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:45 2022] DELETE /identity/v3/users/7a49246b355d43c4ba5c241e19bde45e => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:46.036162 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f6927-aafe-422a-bcb7-c103fff4a6c3 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:46.038185 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-c94f6927-aafe-422a-bcb7-c103fff4a6c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:46.070348 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 932/1890] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 14:09:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:46.138951 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:46.158933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f6927-aafe-422a-bcb7-c103fff4a6c3 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:46.167105 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce6156-c380-404c-aa0b-7fde068e55b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:46.168518 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce6156-c380-404c-aa0b-7fde068e55b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:46.168972 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce6156-c380-404c-aa0b-7fde068e55b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:46.169077 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce6156-c380-404c-aa0b-7fde068e55b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:46.169910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce6156-c380-404c-aa0b-7fde068e55b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:46.170149 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf9e13a-89ab-4bd2-82c7-cf5f2613a0c6 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:46.170314 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce6156-c380-404c-aa0b-7fde068e55b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:46.171642 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0cf9e13a-89ab-4bd2-82c7-cf5f2613a0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:46.172250 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce6156-c380-404c-aa0b-7fde068e55b6 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:46.198823 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 959/1891] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:09:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:46.200850 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce6156-c380-404c-aa0b-7fde068e55b6 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:46.202402 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf9e13a-89ab-4bd2-82c7-cf5f2613a0c6 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:46.214529 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce6156-c380-404c-aa0b-7fde068e55b6 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:46.221377 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49726b39-df44-429b-9a36-24c1a4eeb0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:46.222226 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49726b39-df44-429b-9a36-24c1a4eeb0fb admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:46.222520 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49726b39-df44-429b-9a36-24c1a4eeb0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:46.222796 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49726b39-df44-429b-9a36-24c1a4eeb0fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:46.223805 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49726b39-df44-429b-9a36-24c1a4eeb0fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:46.225793 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49726b39-df44-429b-9a36-24c1a4eeb0fb admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:46.261990 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 960/1892] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:46 2022] POST /identity/v3/projects => generated 389 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:46.266309 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49726b39-df44-429b-9a36-24c1a4eeb0fb admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:46.305349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9e904-f38f-40e7-850d-8563d11322b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:46.306073 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9e904-f38f-40e7-850d-8563d11322b5 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:46.306427 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9e904-f38f-40e7-850d-8563d11322b5 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:46.306783 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9e904-f38f-40e7-850d-8563d11322b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:46.329559 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f9e904-f38f-40e7-850d-8563d11322b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:46.332292 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f9e904-f38f-40e7-850d-8563d11322b5 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:46.340014 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:46.375761 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49726b39-df44-429b-9a36-24c1a4eeb0fb admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:46.377746 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 933/1893] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:09:46.381507 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ef0e04-a608-4bb7-893c-1c8efc94d57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:46.382803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef0e04-a608-4bb7-893c-1c8efc94d57c admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:46.383099 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef0e04-a608-4bb7-893c-1c8efc94d57c admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:46.383433 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef0e04-a608-4bb7-893c-1c8efc94d57c admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:46.384112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ef0e04-a608-4bb7-893c-1c8efc94d57c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:46.387895 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ef0e04-a608-4bb7-893c-1c8efc94d57c admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:46.428813 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 961/1894] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:46 2022] POST /identity/v3/users => generated 295 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:46.432750 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9e904-f38f-40e7-850d-8563d11322b5 service nova] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:46.446141 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9e904-f38f-40e7-850d-8563d11322b5 service nova] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:46.454396 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f0843-e3c1-4e25-ab1b-d5f1e5f2d517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:46.455183 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f0843-e3c1-4e25-ab1b-d5f1e5f2d517 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:46.456074 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f0843-e3c1-4e25-ab1b-d5f1e5f2d517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:46.456074 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f0843-e3c1-4e25-ab1b-d5f1e5f2d517 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:46.456640 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20f0843-e3c1-4e25-ab1b-d5f1e5f2d517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:46.464949 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20f0843-e3c1-4e25-ab1b-d5f1e5f2d517 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:46.484622 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 934/1895] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:46 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:46.487034 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ef0e04-a608-4bb7-893c-1c8efc94d57c admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:46.499116 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ef0e04-a608-4bb7-893c-1c8efc94d57c admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:46.505092 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5349a4-471f-49d7-98e1-4b8a315dccab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:46.505898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5349a4-471f-49d7-98e1-4b8a315dccab admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:46.506272 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5349a4-471f-49d7-98e1-4b8a315dccab admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:46.506732 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5349a4-471f-49d7-98e1-4b8a315dccab admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:46.507971 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5349a4-471f-49d7-98e1-4b8a315dccab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:46.509924 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5349a4-471f-49d7-98e1-4b8a315dccab admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:46.522882 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 962/1896] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:46.527190 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f0843-e3c1-4e25-ab1b-d5f1e5f2d517 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:46.532908 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5349a4-471f-49d7-98e1-4b8a315dccab admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:46.536657 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-3a5349a4-471f-49d7-98e1-4b8a315dccab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:46.546865 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f0843-e3c1-4e25-ab1b-d5f1e5f2d517 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:46.551825 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed03796-8db3-4a85-96c2-0aed12da6a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:46.552535 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed03796-8db3-4a85-96c2-0aed12da6a85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:46.552824 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed03796-8db3-4a85-96c2-0aed12da6a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:46.553103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed03796-8db3-4a85-96c2-0aed12da6a85 admin admin] PATH_INFO: `/v3/projects/9a0ea2d7a35047fca10dc52b64f8ecbe/users/df9c59addb274ab097b35fb9fd4aa6ac/roles/926b9f36c24847c582c84af904250abc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:46.557754 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5349a4-471f-49d7-98e1-4b8a315dccab admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:46.562847 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1880c1-2dd1-4162-b1fb-39cbdcdc7481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:46.563590 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1880c1-2dd1-4162-b1fb-39cbdcdc7481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:46.564715 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1880c1-2dd1-4162-b1fb-39cbdcdc7481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:46.565085 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1880c1-2dd1-4162-b1fb-39cbdcdc7481 admin admin] PATH_INFO: `/v3/projects/0f4160c3d97743fb91bcb541e5843ac9` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:46.566856 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1880c1-2dd1-4162-b1fb-39cbdcdc7481 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f4160c3d97743fb91bcb541e5843ac9)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:46.575427 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1880c1-2dd1-4162-b1fb-39cbdcdc7481 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:46.586516 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed03796-8db3-4a85-96c2-0aed12da6a85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a0ea2d7a35047fca10dc52b64f8ecbe, user_id=df9c59addb274ab097b35fb9fd4aa6ac, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:46.589810 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed03796-8db3-4a85-96c2-0aed12da6a85 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:46.608540 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 935/1897] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:46 2022] PUT /identity/v3/projects/9a0ea2d7a35047fca10dc52b64f8ecbe/users/df9c59addb274ab097b35fb9fd4aa6ac/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:46.628344 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a53281f-c5ef-4e66-b963-814f5b9ca0f6 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:46.628847 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a53281f-c5ef-4e66-b963-814f5b9ca0f6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:46.630440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a53281f-c5ef-4e66-b963-814f5b9ca0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:46.642081 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-4a1880c1-2dd1-4162-b1fb-39cbdcdc7481 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f4160c3d97743fb91bcb541e5843ac9', 'request_id': 'req-4a1880c1-2dd1-4162-b1fb-39cbdcdc7481'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:46.650294 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-4a1880c1-2dd1-4162-b1fb-39cbdcdc7481 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f4160c3d97743fb91bcb541e5843ac9', 'request_id': 'req-4a1880c1-2dd1-4162-b1fb-39cbdcdc7481'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:46.657877 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-4a1880c1-2dd1-4162-b1fb-39cbdcdc7481 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f4160c3d97743fb91bcb541e5843ac9', 'request_id': 'req-4a1880c1-2dd1-4162-b1fb-39cbdcdc7481'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:46.673307 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-4a1880c1-2dd1-4162-b1fb-39cbdcdc7481 admin admin] The token cache is being invalidate because project 0f4160c3d97743fb91bcb541e5843ac9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:46.673601 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-4a1880c1-2dd1-4162-b1fb-39cbdcdc7481 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:46.674321 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-4a1880c1-2dd1-4162-b1fb-39cbdcdc7481 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:46.676554 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 963/1898] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:46 2022] DELETE /identity/v3/projects/0f4160c3d97743fb91bcb541e5843ac9 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:46.677295 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-1a53281f-c5ef-4e66-b963-814f5b9ca0f6 None None] MFA Rules not processed for user `df9c59addb274ab097b35fb9fd4aa6ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:46.678508 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1a53281f-c5ef-4e66-b963-814f5b9ca0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:46.681431 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53044e-314f-4db9-97aa-60b7fda05684 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:46.682008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53044e-314f-4db9-97aa-60b7fda05684 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:46.683282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53044e-314f-4db9-97aa-60b7fda05684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:46.711689 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-4e53044e-314f-4db9-97aa-60b7fda05684 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:46.713727 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-4e53044e-314f-4db9-97aa-60b7fda05684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:46.742831 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:46.798654 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 936/1899] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:46.803289 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a53281f-c5ef-4e66-b963-814f5b9ca0f6 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:46.805011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1a53281f-c5ef-4e66-b963-814f5b9ca0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:46.829314 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 964/1900] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:46.837895 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825332b-91ab-4ecd-8055-c001b4541c96 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:46.838320 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825332b-91ab-4ecd-8055-c001b4541c96 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:46.838786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825332b-91ab-4ecd-8055-c001b4541c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:46.865890 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-e825332b-91ab-4ecd-8055-c001b4541c96 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:46.867422 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-e825332b-91ab-4ecd-8055-c001b4541c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:46.878271 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 965/1901] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:46.882883 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825332b-91ab-4ecd-8055-c001b4541c96 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:46.889251 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a53281f-c5ef-4e66-b963-814f5b9ca0f6 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:46.892847 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825332b-91ab-4ecd-8055-c001b4541c96 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:46.894652 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9adc3b-9c65-4b0a-8e7f-e91bf747f19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:46.895287 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9adc3b-9c65-4b0a-8e7f-e91bf747f19e admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:46.895576 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9adc3b-9c65-4b0a-8e7f-e91bf747f19e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:46.895854 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9adc3b-9c65-4b0a-8e7f-e91bf747f19e admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:46.896581 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9adc3b-9c65-4b0a-8e7f-e91bf747f19e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:46.899098 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9adc3b-9c65-4b0a-8e7f-e91bf747f19e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:46.900650 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b74575-2503-4c7a-9a14-95370ef9a628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:46.901512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b74575-2503-4c7a-9a14-95370ef9a628 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:46.901907 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b74575-2503-4c7a-9a14-95370ef9a628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:46.902294 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b74575-2503-4c7a-9a14-95370ef9a628 admin admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:46.903077 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b74575-2503-4c7a-9a14-95370ef9a628 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:46.903484 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b74575-2503-4c7a-9a14-95370ef9a628 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:46.905586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b74575-2503-4c7a-9a14-95370ef9a628 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:46.921129 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 966/1902] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:09:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:46.925707 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b74575-2503-4c7a-9a14-95370ef9a628 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:46.939603 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 937/1903] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:46 2022] POST /identity/v3/projects => generated 389 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:09:46.941752 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b74575-2503-4c7a-9a14-95370ef9a628 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:46.944943 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9adc3b-9c65-4b0a-8e7f-e91bf747f19e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:46.999676 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:47.022761 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a62d3d4-1b5f-41a1-8bff-a16b322336b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:47.023662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62d3d4-1b5f-41a1-8bff-a16b322336b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:47.023662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62d3d4-1b5f-41a1-8bff-a16b322336b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:47.023959 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62d3d4-1b5f-41a1-8bff-a16b322336b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:47.024641 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a62d3d4-1b5f-41a1-8bff-a16b322336b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:47.024770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9adc3b-9c65-4b0a-8e7f-e91bf747f19e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:47.026562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a62d3d4-1b5f-41a1-8bff-a16b322336b6 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:47.031007 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f0c4a-ee57-4166-969d-1c2d25f3656c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:47.031886 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f0c4a-ee57-4166-969d-1c2d25f3656c admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:47.032035 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f0c4a-ee57-4166-969d-1c2d25f3656c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:47.032328 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f0c4a-ee57-4166-969d-1c2d25f3656c admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:47.033054 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539f0c4a-ee57-4166-969d-1c2d25f3656c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:47.034991 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539f0c4a-ee57-4166-969d-1c2d25f3656c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:47.062611 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 938/1904] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:46 2022] POST /identity/v3/users => generated 301 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:09:47.062811 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 967/1905] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:46 2022] POST /identity/v3/projects => generated 369 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:47.064158 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f0c4a-ee57-4166-969d-1c2d25f3656c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:47.066302 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a62d3d4-1b5f-41a1-8bff-a16b322336b6 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:47.075951 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f0c4a-ee57-4166-969d-1c2d25f3656c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:47.141179 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3330b96-057f-4e6a-893c-633ae5bc294d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:47.141804 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3330b96-057f-4e6a-893c-633ae5bc294d admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:47.142086 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3330b96-057f-4e6a-893c-633ae5bc294d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:47.142363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3330b96-057f-4e6a-893c-633ae5bc294d admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:47.142456 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a62d3d4-1b5f-41a1-8bff-a16b322336b6 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:47.143062 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3330b96-057f-4e6a-893c-633ae5bc294d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:47.144713 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3330b96-057f-4e6a-893c-633ae5bc294d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:47.149189 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd610c-9364-41ec-bb38-c81dbfa130ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:47.149921 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd610c-9364-41ec-bb38-c81dbfa130ba admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:47.150728 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd610c-9364-41ec-bb38-c81dbfa130ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:47.151014 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd610c-9364-41ec-bb38-c81dbfa130ba admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:47.151684 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdd610c-9364-41ec-bb38-c81dbfa130ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:47.153787 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdd610c-9364-41ec-bb38-c81dbfa130ba admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:47.154934 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 939/1906] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:47 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:47.160513 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3330b96-057f-4e6a-893c-633ae5bc294d admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:47.177169 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3330b96-057f-4e6a-893c-633ae5bc294d admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:47.183518 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 968/1907] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:47 2022] POST /identity/v3/users => generated 285 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:09:47.184643 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a210ed3f-d4ed-4f27-a0df-a69f2a9da5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:47.184893 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd610c-9364-41ec-bb38-c81dbfa130ba admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:47.185838 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210ed3f-d4ed-4f27-a0df-a69f2a9da5a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:47.186220 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210ed3f-d4ed-4f27-a0df-a69f2a9da5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:47.186561 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210ed3f-d4ed-4f27-a0df-a69f2a9da5a7 admin admin] PATH_INFO: `/v3/projects/631839aa38a644e09cc77a269c09b037/users/ba2811bd9f1f497c956e859707cd5e2a/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:47.195724 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd610c-9364-41ec-bb38-c81dbfa130ba admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:47.200554 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba52011-8972-4109-b895-b380cc142274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:47.201147 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba52011-8972-4109-b895-b380cc142274 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:47.201581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba52011-8972-4109-b895-b380cc142274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:47.201873 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba52011-8972-4109-b895-b380cc142274 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:47.202591 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba52011-8972-4109-b895-b380cc142274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:47.203104 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a210ed3f-d4ed-4f27-a0df-a69f2a9da5a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=631839aa38a644e09cc77a269c09b037, user_id=ba2811bd9f1f497c956e859707cd5e2a, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:47.204751 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba52011-8972-4109-b895-b380cc142274 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:47.205952 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a210ed3f-d4ed-4f27-a0df-a69f2a9da5a7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:47.216544 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 969/1908] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:47.220128 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba52011-8972-4109-b895-b380cc142274 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:47.222709 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 940/1909] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:47 2022] PUT /identity/v3/projects/631839aa38a644e09cc77a269c09b037/users/ba2811bd9f1f497c956e859707cd5e2a/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:47.227064 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a210ed3f-d4ed-4f27-a0df-a69f2a9da5a7 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:47.232666 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba52011-8972-4109-b895-b380cc142274 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:47.238971 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:47.310351 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ece2a1-2e72-4b5a-bc79-9c7ed1db18f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:47.311557 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ece2a1-2e72-4b5a-bc79-9c7ed1db18f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:47.311557 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ece2a1-2e72-4b5a-bc79-9c7ed1db18f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:47.311754 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ece2a1-2e72-4b5a-bc79-9c7ed1db18f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:47.312398 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ece2a1-2e72-4b5a-bc79-9c7ed1db18f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:47.314173 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ece2a1-2e72-4b5a-bc79-9c7ed1db18f2 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:47.315221 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a210ed3f-d4ed-4f27-a0df-a69f2a9da5a7 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:47.322272 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e96c8-5cb0-4a34-8214-6e9ea1a79509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:47.322902 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e96c8-5cb0-4a34-8214-6e9ea1a79509 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:47.323235 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e96c8-5cb0-4a34-8214-6e9ea1a79509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:47.323524 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e96c8-5cb0-4a34-8214-6e9ea1a79509 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:47.324443 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28e96c8-5cb0-4a34-8214-6e9ea1a79509 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:47.326116 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28e96c8-5cb0-4a34-8214-6e9ea1a79509 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:47.326669 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 970/1910] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:47 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:47.331125 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ece2a1-2e72-4b5a-bc79-9c7ed1db18f2 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:47.343430 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 941/1911] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:47 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:47.344238 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ece2a1-2e72-4b5a-bc79-9c7ed1db18f2 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:47.348089 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e96c8-5cb0-4a34-8214-6e9ea1a79509 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:47.352974 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c7b9a-633a-4b3e-a1e2-0bbad58ebd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:47.353668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c7b9a-633a-4b3e-a1e2-0bbad58ebd40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:47.354092 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c7b9a-633a-4b3e-a1e2-0bbad58ebd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:47.354381 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c7b9a-633a-4b3e-a1e2-0bbad58ebd40 admin admin] PATH_INFO: `/v3/projects/86608df68045413981929226e1defe1c/users/80a743e783684d55a572c62368ba7023/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:47.362200 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e96c8-5cb0-4a34-8214-6e9ea1a79509 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:47.367926 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d96469-8f27-4b95-bbe0-84738662c62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:47.368682 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d96469-8f27-4b95-bbe0-84738662c62f admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:47.368964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d96469-8f27-4b95-bbe0-84738662c62f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:47.369416 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d96469-8f27-4b95-bbe0-84738662c62f admin admin] PATH_INFO: `/v3/projects/631839aa38a644e09cc77a269c09b037/users/ba2811bd9f1f497c956e859707cd5e2a/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:47.372275 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d96469-8f27-4b95-bbe0-84738662c62f admin admin] RBAC: Authorizing `identity:create_grant(project_id=631839aa38a644e09cc77a269c09b037, user_id=ba2811bd9f1f497c956e859707cd5e2a, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:47.375645 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d96469-8f27-4b95-bbe0-84738662c62f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:47.380342 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c7b9a-633a-4b3e-a1e2-0bbad58ebd40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86608df68045413981929226e1defe1c, user_id=80a743e783684d55a572c62368ba7023, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:47.386199 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c7b9a-633a-4b3e-a1e2-0bbad58ebd40 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:47.403314 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 971/1912] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:47 2022] PUT /identity/v3/projects/86608df68045413981929226e1defe1c/users/80a743e783684d55a572c62368ba7023/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:47.406210 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 942/1913] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:47 2022] PUT /identity/v3/projects/631839aa38a644e09cc77a269c09b037/users/ba2811bd9f1f497c956e859707cd5e2a/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:47.417110 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896732a2-f914-4188-8be2-4a4b6e1c380d None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:47.417396 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896732a2-f914-4188-8be2-4a4b6e1c380d None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:47.417671 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896732a2-f914-4188-8be2-4a4b6e1c380d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:47.418487 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8b076-4fee-409d-ac05-895da9456db6 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:47.418843 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8b076-4fee-409d-ac05-895da9456db6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:47.419212 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8b076-4fee-409d-ac05-895da9456db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:47.449336 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-896732a2-f914-4188-8be2-4a4b6e1c380d None None] MFA Rules not processed for user `80a743e783684d55a572c62368ba7023`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:47.450966 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-896732a2-f914-4188-8be2-4a4b6e1c380d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:47.453533 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c0d8b076-4fee-409d-ac05-895da9456db6 None None] MFA Rules not processed for user `ba2811bd9f1f497c956e859707cd5e2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:47.455044 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c0d8b076-4fee-409d-ac05-895da9456db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:47.465985 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:47.535839 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 972/1914] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:47 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:47.540948 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896732a2-f914-4188-8be2-4a4b6e1c380d None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:47.550999 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 943/1915] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:47 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:47.564072 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8b076-4fee-409d-ac05-895da9456db6 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:47.565728 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c0d8b076-4fee-409d-ac05-895da9456db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:47.632325 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8b076-4fee-409d-ac05-895da9456db6 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:47.634788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896732a2-f914-4188-8be2-4a4b6e1c380d None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:47.635666 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2407f56c-8fbd-47f3-be35-e17f9961f364 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:47.636299 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407f56c-8fbd-47f3-be35-e17f9961f364 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:47.636684 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407f56c-8fbd-47f3-be35-e17f9961f364 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:47.636953 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407f56c-8fbd-47f3-be35-e17f9961f364 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:47.642218 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b6354-ae06-432f-8323-1012d168d8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:47.643743 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b6354-ae06-432f-8323-1012d168d8ba admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:47.644282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b6354-ae06-432f-8323-1012d168d8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:47.644774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b6354-ae06-432f-8323-1012d168d8ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:47.645457 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b6354-ae06-432f-8323-1012d168d8ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:47.646743 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2407f56c-8fbd-47f3-be35-e17f9961f364 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:47.647995 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b6354-ae06-432f-8323-1012d168d8ba admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:47.648820 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2407f56c-8fbd-47f3-be35-e17f9961f364 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:47.661914 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 944/1916] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:47 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:09:47.664549 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:47.687854 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 973/1917] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:47 2022] POST /identity/v3/projects => generated 369 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:09:47.687961 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2407f56c-8fbd-47f3-be35-e17f9961f364 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:47.702350 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2407f56c-8fbd-47f3-be35-e17f9961f364 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:47.774535 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14347d66-14b9-4c2a-a14d-3ea3463080c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:47.774885 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14347d66-14b9-4c2a-a14d-3ea3463080c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:47.775169 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14347d66-14b9-4c2a-a14d-3ea3463080c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:47.775491 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14347d66-14b9-4c2a-a14d-3ea3463080c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:47.776145 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14347d66-14b9-4c2a-a14d-3ea3463080c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:47.778067 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14347d66-14b9-4c2a-a14d-3ea3463080c7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:47.815181 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 945/1918] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:47 2022] POST /identity/v3/users => generated 291 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:47.821219 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b6354-ae06-432f-8323-1012d168d8ba admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:47.836096 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14347d66-14b9-4c2a-a14d-3ea3463080c7 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:47.837965 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-14347d66-14b9-4c2a-a14d-3ea3463080c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:47.842658 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b6354-ae06-432f-8323-1012d168d8ba admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:47.848268 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f50c9-53a3-4e38-8cc8-630feafe3dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:47.849102 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f50c9-53a3-4e38-8cc8-630feafe3dd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:47.849486 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f50c9-53a3-4e38-8cc8-630feafe3dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:47.849887 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f50c9-53a3-4e38-8cc8-630feafe3dd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:47.850204 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14347d66-14b9-4c2a-a14d-3ea3463080c7 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:47.850969 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7f50c9-53a3-4e38-8cc8-630feafe3dd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:47.852726 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7f50c9-53a3-4e38-8cc8-630feafe3dd1 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:47.855303 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdebda7-10d6-4bfe-b07f-5d2292901013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:47.855949 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdebda7-10d6-4bfe-b07f-5d2292901013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:47.856236 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdebda7-10d6-4bfe-b07f-5d2292901013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:47.856555 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdebda7-10d6-4bfe-b07f-5d2292901013 admin admin] PATH_INFO: `/v3/users/9a3e541656b342698f0efb9fe1ee0e07` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:47.857736 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdebda7-10d6-4bfe-b07f-5d2292901013 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a3e541656b342698f0efb9fe1ee0e07)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:47.860117 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdebda7-10d6-4bfe-b07f-5d2292901013 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:47.863671 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 974/1919] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:47 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:47.864714 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f50c9-53a3-4e38-8cc8-630feafe3dd1 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:47.875731 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f50c9-53a3-4e38-8cc8-630feafe3dd1 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:47.880699 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c929c6-d609-42cb-94ca-6c2284b43de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:47.882008 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c929c6-d609-42cb-94ca-6c2284b43de6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:47.882346 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c929c6-d609-42cb-94ca-6c2284b43de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:47.882633 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c929c6-d609-42cb-94ca-6c2284b43de6 admin admin] PATH_INFO: `/v3/users/df9c59addb274ab097b35fb9fd4aa6ac` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:47.883934 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c929c6-d609-42cb-94ca-6c2284b43de6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df9c59addb274ab097b35fb9fd4aa6ac)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:47.886076 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c929c6-d609-42cb-94ca-6c2284b43de6 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:47.905681 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:47.926992 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5fdebda7-10d6-4bfe-b07f-5d2292901013 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a3e541656b342698f0efb9fe1ee0e07'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:47.939720 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5fdebda7-10d6-4bfe-b07f-5d2292901013 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a3e541656b342698f0efb9fe1ee0e07'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:47.940783 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5fdebda7-10d6-4bfe-b07f-5d2292901013 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a3e541656b342698f0efb9fe1ee0e07'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:47.949683 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-54c929c6-d609-42cb-94ca-6c2284b43de6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df9c59addb274ab097b35fb9fd4aa6ac'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:47.951829 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5fdebda7-10d6-4bfe-b07f-5d2292901013 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a3e541656b342698f0efb9fe1ee0e07'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:47.952108 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-54c929c6-d609-42cb-94ca-6c2284b43de6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df9c59addb274ab097b35fb9fd4aa6ac'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:47.953687 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5fdebda7-10d6-4bfe-b07f-5d2292901013 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a3e541656b342698f0efb9fe1ee0e07'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:47.968965 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-54c929c6-d609-42cb-94ca-6c2284b43de6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df9c59addb274ab097b35fb9fd4aa6ac'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:47.981717 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-54c929c6-d609-42cb-94ca-6c2284b43de6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df9c59addb274ab097b35fb9fd4aa6ac'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:47.982410 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 946/1920] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:47 2022] DELETE /identity/v3/users/9a3e541656b342698f0efb9fe1ee0e07 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:47.983287 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdebda7-10d6-4bfe-b07f-5d2292901013 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:47.984157 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-54c929c6-d609-42cb-94ca-6c2284b43de6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df9c59addb274ab097b35fb9fd4aa6ac'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:47.984826 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5fdebda7-10d6-4bfe-b07f-5d2292901013 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:48.002769 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 975/1921] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:47 2022] DELETE /identity/v3/users/df9c59addb274ab097b35fb9fd4aa6ac => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:48.005982 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8598265-242d-402c-a0f4-10f1789a099d None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:48.006204 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8598265-242d-402c-a0f4-10f1789a099d None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:48.007035 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8598265-242d-402c-a0f4-10f1789a099d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:48.032095 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-c8598265-242d-402c-a0f4-10f1789a099d None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:48.033272 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-c8598265-242d-402c-a0f4-10f1789a099d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:48.079173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdebda7-10d6-4bfe-b07f-5d2292901013 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:48.114723 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 976/1922] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:48.120051 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80303a6-a7b1-4b4a-bf0b-da1ab15a75e6 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:48.120417 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80303a6-a7b1-4b4a-bf0b-da1ab15a75e6 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:48.120734 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80303a6-a7b1-4b4a-bf0b-da1ab15a75e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:48.141089 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-e80303a6-a7b1-4b4a-bf0b-da1ab15a75e6 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:48.142396 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-e80303a6-a7b1-4b4a-bf0b-da1ab15a75e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:48.150781 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 977/1923] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:48.151476 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80303a6-a7b1-4b4a-bf0b-da1ab15a75e6 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:48.152717 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-e80303a6-a7b1-4b4a-bf0b-da1ab15a75e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:48.156821 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996e79ef-3472-469c-bc63-9ce34f35bf4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:48.157875 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e79ef-3472-469c-bc63-9ce34f35bf4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:48.158179 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e79ef-3472-469c-bc63-9ce34f35bf4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:48.158467 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e79ef-3472-469c-bc63-9ce34f35bf4e admin admin] PATH_INFO: `/v3/projects/41482dd8164d47f0bbc52b813efe5e89/users/40c487f7fe5c4c929ae08662280577bb/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:48.177316 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996e79ef-3472-469c-bc63-9ce34f35bf4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=41482dd8164d47f0bbc52b813efe5e89, user_id=40c487f7fe5c4c929ae08662280577bb, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:48.180469 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996e79ef-3472-469c-bc63-9ce34f35bf4e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:48.198247 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 947/1924] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:47 2022] PUT /identity/v3/projects/41482dd8164d47f0bbc52b813efe5e89/users/40c487f7fe5c4c929ae08662280577bb/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:48.199262 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996e79ef-3472-469c-bc63-9ce34f35bf4e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:48.202331 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-996e79ef-3472-469c-bc63-9ce34f35bf4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:48.223551 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:48.229689 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80303a6-a7b1-4b4a-bf0b-da1ab15a75e6 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:48.259630 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:48.277222 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996e79ef-3472-469c-bc63-9ce34f35bf4e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:48.282286 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5d732-e978-4818-84bd-c795b4938143 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:48.282848 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5d732-e978-4818-84bd-c795b4938143 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:48.283133 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5d732-e978-4818-84bd-c795b4938143 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:48.283826 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5d732-e978-4818-84bd-c795b4938143 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:48.293883 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5997c93-65b4-4c6f-9bda-0c1ba111e24c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:48.294500 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5997c93-65b4-4c6f-9bda-0c1ba111e24c service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:48.294776 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5997c93-65b4-4c6f-9bda-0c1ba111e24c service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:48.295050 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5997c93-65b4-4c6f-9bda-0c1ba111e24c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:48.359298 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b5d732-e978-4818-84bd-c795b4938143 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:48.361329 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b5d732-e978-4818-84bd-c795b4938143 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:48.372914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5997c93-65b4-4c6f-9bda-0c1ba111e24c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:48.373512 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 948/1925] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:48.374346 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5d732-e978-4818-84bd-c795b4938143 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:48.374537 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5997c93-65b4-4c6f-9bda-0c1ba111e24c service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:48.375902 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a4b5d732-e978-4818-84bd-c795b4938143 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:48.385657 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 978/1926] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:48.390721 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5d732-e978-4818-84bd-c795b4938143 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:48.404623 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4560e54c-f386-4824-9cc4-625a2791eaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:48.405449 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560e54c-f386-4824-9cc4-625a2791eaec admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:48.405893 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560e54c-f386-4824-9cc4-625a2791eaec admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:48.406263 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560e54c-f386-4824-9cc4-625a2791eaec admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:48.407372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4560e54c-f386-4824-9cc4-625a2791eaec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:48.410474 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4560e54c-f386-4824-9cc4-625a2791eaec admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:48.434250 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5997c93-65b4-4c6f-9bda-0c1ba111e24c service neutron] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:48.440820 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 949/1927] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:48 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:48.448914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5997c93-65b4-4c6f-9bda-0c1ba111e24c service neutron] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:48.456438 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276313f-b683-4588-baf0-2ce45fb0ec0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:48.457403 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276313f-b683-4588-baf0-2ce45fb0ec0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:48.457694 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276313f-b683-4588-baf0-2ce45fb0ec0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:48.457979 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276313f-b683-4588-baf0-2ce45fb0ec0e admin admin] PATH_INFO: `/v3/projects/41482dd8164d47f0bbc52b813efe5e89/users/40c487f7fe5c4c929ae08662280577bb/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:48.461610 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1276313f-b683-4588-baf0-2ce45fb0ec0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=41482dd8164d47f0bbc52b813efe5e89, user_id=40c487f7fe5c4c929ae08662280577bb, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:48.465113 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1276313f-b683-4588-baf0-2ce45fb0ec0e admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:48.478908 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 979/1928] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:48 2022] PUT /identity/v3/projects/41482dd8164d47f0bbc52b813efe5e89/users/40c487f7fe5c4c929ae08662280577bb/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:48.490490 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72718805-8851-4027-9505-d13993f56e7f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:48.490859 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72718805-8851-4027-9505-d13993f56e7f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:48.491102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72718805-8851-4027-9505-d13993f56e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:48.524531 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-72718805-8851-4027-9505-d13993f56e7f None None] MFA Rules not processed for user `40c487f7fe5c4c929ae08662280577bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:48.525698 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-72718805-8851-4027-9505-d13993f56e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:48.703156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 950/1929] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:48 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:48.716417 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276313f-b683-4588-baf0-2ce45fb0ec0e admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:48.718690 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-1276313f-b683-4588-baf0-2ce45fb0ec0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:48.794213 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276313f-b683-4588-baf0-2ce45fb0ec0e admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:48.803093 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47758999-2f7b-4a17-b26a-3183b7f05f73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:48.803893 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47758999-2f7b-4a17-b26a-3183b7f05f73 service cinder] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:48.804334 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47758999-2f7b-4a17-b26a-3183b7f05f73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:48.805255 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47758999-2f7b-4a17-b26a-3183b7f05f73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:48.825290 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47758999-2f7b-4a17-b26a-3183b7f05f73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:48.826829 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47758999-2f7b-4a17-b26a-3183b7f05f73 service cinder] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:48.840274 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 980/1930] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:48 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:48.853176 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72718805-8851-4027-9505-d13993f56e7f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:48.854920 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-72718805-8851-4027-9505-d13993f56e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:48.867508 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47758999-2f7b-4a17-b26a-3183b7f05f73 service cinder] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:48.876744 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-47758999-2f7b-4a17-b26a-3183b7f05f73 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:48.943524 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:48.963280 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72718805-8851-4027-9505-d13993f56e7f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:48.967732 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47758999-2f7b-4a17-b26a-3183b7f05f73 service cinder] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:48.969266 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7ff45a-d01f-494a-8274-4202ef8ac818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:48.970073 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ff45a-d01f-494a-8274-4202ef8ac818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:48.970482 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ff45a-d01f-494a-8274-4202ef8ac818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:48.970761 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ff45a-d01f-494a-8274-4202ef8ac818 admin admin] PATH_INFO: `/v3/projects/9a0ea2d7a35047fca10dc52b64f8ecbe` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:48.971970 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7ff45a-d01f-494a-8274-4202ef8ac818 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a0ea2d7a35047fca10dc52b64f8ecbe)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:48.973948 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c9f964-ce38-401f-91d5-ba0703da540c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:48.974098 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7ff45a-d01f-494a-8274-4202ef8ac818 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:48.975522 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9f964-ce38-401f-91d5-ba0703da540c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:48.975918 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9f964-ce38-401f-91d5-ba0703da540c admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:48.976382 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9f964-ce38-401f-91d5-ba0703da540c admin admin] PATH_INFO: `/v3/projects/cf9294f3d32640dbaa215da7172d3c52` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:48.977824 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c9f964-ce38-401f-91d5-ba0703da540c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf9294f3d32640dbaa215da7172d3c52)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:48.980049 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c9f964-ce38-401f-91d5-ba0703da540c admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:48.984319 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:49.028703 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2b7ff45a-d01f-494a-8274-4202ef8ac818 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a0ea2d7a35047fca10dc52b64f8ecbe', 'request_id': 'req-2b7ff45a-d01f-494a-8274-4202ef8ac818'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:49.031908 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d0c9f964-ce38-401f-91d5-ba0703da540c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf9294f3d32640dbaa215da7172d3c52', 'request_id': 'req-d0c9f964-ce38-401f-91d5-ba0703da540c'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:49.036790 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2b7ff45a-d01f-494a-8274-4202ef8ac818 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a0ea2d7a35047fca10dc52b64f8ecbe', 'request_id': 'req-2b7ff45a-d01f-494a-8274-4202ef8ac818'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:49.038643 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d0c9f964-ce38-401f-91d5-ba0703da540c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf9294f3d32640dbaa215da7172d3c52', 'request_id': 'req-d0c9f964-ce38-401f-91d5-ba0703da540c'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:49.044597 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2b7ff45a-d01f-494a-8274-4202ef8ac818 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a0ea2d7a35047fca10dc52b64f8ecbe', 'request_id': 'req-2b7ff45a-d01f-494a-8274-4202ef8ac818'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:49.050056 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d0c9f964-ce38-401f-91d5-ba0703da540c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf9294f3d32640dbaa215da7172d3c52', 'request_id': 'req-d0c9f964-ce38-401f-91d5-ba0703da540c'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:49.055405 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2b7ff45a-d01f-494a-8274-4202ef8ac818 admin admin] The token cache is being invalidate because project 9a0ea2d7a35047fca10dc52b64f8ecbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:49.055724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2b7ff45a-d01f-494a-8274-4202ef8ac818 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:49.056522 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2b7ff45a-d01f-494a-8274-4202ef8ac818 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:49.058327 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 951/1931] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:48 2022] DELETE /identity/v3/projects/9a0ea2d7a35047fca10dc52b64f8ecbe => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:49.059359 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7ff45a-d01f-494a-8274-4202ef8ac818 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:49.060772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2b7ff45a-d01f-494a-8274-4202ef8ac818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:49.065329 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d0c9f964-ce38-401f-91d5-ba0703da540c admin admin] The token cache is being invalidate because project cf9294f3d32640dbaa215da7172d3c52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:49.065653 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d0c9f964-ce38-401f-91d5-ba0703da540c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:49.066324 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d0c9f964-ce38-401f-91d5-ba0703da540c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:49.068029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 981/1932] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:48 2022] DELETE /identity/v3/projects/cf9294f3d32640dbaa215da7172d3c52 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:49.069391 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c9f964-ce38-401f-91d5-ba0703da540c admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:49.070682 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-d0c9f964-ce38-401f-91d5-ba0703da540c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:49.126624 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7ff45a-d01f-494a-8274-4202ef8ac818 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:49.150242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c9f964-ce38-401f-91d5-ba0703da540c admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:49.155239 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa079bb-4299-4efc-9b79-6d2b38b4fef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:49.155810 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa079bb-4299-4efc-9b79-6d2b38b4fef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:49.156095 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa079bb-4299-4efc-9b79-6d2b38b4fef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:49.156377 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa079bb-4299-4efc-9b79-6d2b38b4fef4 admin admin] PATH_INFO: `/v3/users/ba2811bd9f1f497c956e859707cd5e2a` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:49.157715 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa079bb-4299-4efc-9b79-6d2b38b4fef4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba2811bd9f1f497c956e859707cd5e2a)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:49.159675 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa079bb-4299-4efc-9b79-6d2b38b4fef4 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:49.191881 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d3d165-069b-42b9-ad74-c86f8aad52c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:49.192568 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3d165-069b-42b9-ad74-c86f8aad52c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:49.192854 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3d165-069b-42b9-ad74-c86f8aad52c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:49.193128 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3d165-069b-42b9-ad74-c86f8aad52c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:49.195499 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-46d3d165-069b-42b9-ad74-c86f8aad52c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:49.217905 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-0fa079bb-4299-4efc-9b79-6d2b38b4fef4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba2811bd9f1f497c956e859707cd5e2a'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:49.218702 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-0fa079bb-4299-4efc-9b79-6d2b38b4fef4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba2811bd9f1f497c956e859707cd5e2a'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:49.233138 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-0fa079bb-4299-4efc-9b79-6d2b38b4fef4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba2811bd9f1f497c956e859707cd5e2a'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:49.241705 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-0fa079bb-4299-4efc-9b79-6d2b38b4fef4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba2811bd9f1f497c956e859707cd5e2a'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:49.242869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-0fa079bb-4299-4efc-9b79-6d2b38b4fef4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba2811bd9f1f497c956e859707cd5e2a'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:49.248949 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:49.262603 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d3d165-069b-42b9-ad74-c86f8aad52c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:49.263050 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 982/1933] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:49 2022] DELETE /identity/v3/users/ba2811bd9f1f497c956e859707cd5e2a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:49.264043 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa079bb-4299-4efc-9b79-6d2b38b4fef4 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:49.264623 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d3d165-069b-42b9-ad74-c86f8aad52c0 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:49.265856 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-0fa079bb-4299-4efc-9b79-6d2b38b4fef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:49.278900 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 952/1934] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:49 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:49.377512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa079bb-4299-4efc-9b79-6d2b38b4fef4 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:49.385931 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2f6a2-eb21-43f0-bdbf-b67b491c2108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:49.387125 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2f6a2-eb21-43f0-bdbf-b67b491c2108 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:49.387442 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2f6a2-eb21-43f0-bdbf-b67b491c2108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:49.387733 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2f6a2-eb21-43f0-bdbf-b67b491c2108 admin admin] PATH_INFO: `/v3/users/9a8fc4c4871e4511b8cb7556d6861463` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:49.389051 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2f6a2-eb21-43f0-bdbf-b67b491c2108 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a8fc4c4871e4511b8cb7556d6861463)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:49.392978 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2f6a2-eb21-43f0-bdbf-b67b491c2108 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:49.473589 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-b7a2f6a2-eb21-43f0-bdbf-b67b491c2108 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a8fc4c4871e4511b8cb7556d6861463'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:49.474956 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-b7a2f6a2-eb21-43f0-bdbf-b67b491c2108 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a8fc4c4871e4511b8cb7556d6861463'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:49.498235 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-b7a2f6a2-eb21-43f0-bdbf-b67b491c2108 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a8fc4c4871e4511b8cb7556d6861463'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:49.515535 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-b7a2f6a2-eb21-43f0-bdbf-b67b491c2108 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a8fc4c4871e4511b8cb7556d6861463'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:49.516911 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-b7a2f6a2-eb21-43f0-bdbf-b67b491c2108 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a8fc4c4871e4511b8cb7556d6861463'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:49.538875 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 983/1935] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:49 2022] DELETE /identity/v3/users/9a8fc4c4871e4511b8cb7556d6861463 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:49.773341 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d3d165-069b-42b9-ad74-c86f8aad52c0 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:49.807353 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d3d165-069b-42b9-ad74-c86f8aad52c0 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:49.830830 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:49.910484 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445a3a5-5e22-4bd2-b7a3-84326aa9bd12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:49.911245 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445a3a5-5e22-4bd2-b7a3-84326aa9bd12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:49.911987 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445a3a5-5e22-4bd2-b7a3-84326aa9bd12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:49.911987 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445a3a5-5e22-4bd2-b7a3-84326aa9bd12 admin admin] PATH_INFO: `/v3/projects/631839aa38a644e09cc77a269c09b037` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:49.913814 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6445a3a5-5e22-4bd2-b7a3-84326aa9bd12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=631839aa38a644e09cc77a269c09b037)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:49.916929 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6445a3a5-5e22-4bd2-b7a3-84326aa9bd12 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:49.998139 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6445a3a5-5e22-4bd2-b7a3-84326aa9bd12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '631839aa38a644e09cc77a269c09b037', 'request_id': 'req-6445a3a5-5e22-4bd2-b7a3-84326aa9bd12'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:50.018340 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6445a3a5-5e22-4bd2-b7a3-84326aa9bd12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '631839aa38a644e09cc77a269c09b037', 'request_id': 'req-6445a3a5-5e22-4bd2-b7a3-84326aa9bd12'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:50.026283 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6445a3a5-5e22-4bd2-b7a3-84326aa9bd12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '631839aa38a644e09cc77a269c09b037', 'request_id': 'req-6445a3a5-5e22-4bd2-b7a3-84326aa9bd12'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:50.038289 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6445a3a5-5e22-4bd2-b7a3-84326aa9bd12 admin admin] The token cache is being invalidate because project 631839aa38a644e09cc77a269c09b037 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:50.038656 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6445a3a5-5e22-4bd2-b7a3-84326aa9bd12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:50.039492 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6445a3a5-5e22-4bd2-b7a3-84326aa9bd12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:50.041351 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 953/1936] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:49 2022] DELETE /identity/v3/projects/631839aa38a644e09cc77a269c09b037 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:50.050625 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15b13e-ee6c-4f7a-8902-d974c11011f7 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:50.050625 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15b13e-ee6c-4f7a-8902-d974c11011f7 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:50.050625 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15b13e-ee6c-4f7a-8902-d974c11011f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:50.070559 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-fb15b13e-ee6c-4f7a-8902-d974c11011f7 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:50.071796 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-fb15b13e-ee6c-4f7a-8902-d974c11011f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:50.142910 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445a3a5-5e22-4bd2-b7a3-84326aa9bd12 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:50.144823 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6445a3a5-5e22-4bd2-b7a3-84326aa9bd12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:50.151820 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 984/1937] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:50.163121 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ca6e7-4fbc-4f2c-aeb8-8bd7b1ae500e None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:50.164458 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ca6e7-4fbc-4f2c-aeb8-8bd7b1ae500e None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:50.164990 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ca6e7-4fbc-4f2c-aeb8-8bd7b1ae500e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:50.167004 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445a3a5-5e22-4bd2-b7a3-84326aa9bd12 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:50.189642 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a8e54-93cb-4ff8-84b9-e74b3756658d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:50.190309 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a8e54-93cb-4ff8-84b9-e74b3756658d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:50.190592 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a8e54-93cb-4ff8-84b9-e74b3756658d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:50.190984 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a8e54-93cb-4ff8-84b9-e74b3756658d admin admin] PATH_INFO: `/v3/projects/70560ff732f0442494762af8e714fe7e` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:50.192216 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a8e54-93cb-4ff8-84b9-e74b3756658d admin admin] RBAC: Authorizing `identity:delete_project(project_id=70560ff732f0442494762af8e714fe7e)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:50.194937 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a8e54-93cb-4ff8-84b9-e74b3756658d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:50.199087 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-2b4ca6e7-4fbc-4f2c-aeb8-8bd7b1ae500e None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:50.200567 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-2b4ca6e7-4fbc-4f2c-aeb8-8bd7b1ae500e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:50.209779 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 985/1938] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:50.214405 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ca6e7-4fbc-4f2c-aeb8-8bd7b1ae500e None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:50.224514 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ca6e7-4fbc-4f2c-aeb8-8bd7b1ae500e None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:50.229223 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5abd30-63b8-494c-b3a3-43154e513887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:50.229841 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5abd30-63b8-494c-b3a3-43154e513887 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:50.230127 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5abd30-63b8-494c-b3a3-43154e513887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:50.230402 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5abd30-63b8-494c-b3a3-43154e513887 admin admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:50.231101 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5abd30-63b8-494c-b3a3-43154e513887 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:50.231434 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5abd30-63b8-494c-b3a3-43154e513887 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:50.233058 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5abd30-63b8-494c-b3a3-43154e513887 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:50.234628 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:50.246350 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7a1a8e54-93cb-4ff8-84b9-e74b3756658d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70560ff732f0442494762af8e714fe7e', 'request_id': 'req-7a1a8e54-93cb-4ff8-84b9-e74b3756658d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:50.250160 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 986/1939] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:09:50.254524 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5abd30-63b8-494c-b3a3-43154e513887 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:50.255707 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7a1a8e54-93cb-4ff8-84b9-e74b3756658d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70560ff732f0442494762af8e714fe7e', 'request_id': 'req-7a1a8e54-93cb-4ff8-84b9-e74b3756658d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:50.263182 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7a1a8e54-93cb-4ff8-84b9-e74b3756658d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70560ff732f0442494762af8e714fe7e', 'request_id': 'req-7a1a8e54-93cb-4ff8-84b9-e74b3756658d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:50.265769 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5abd30-63b8-494c-b3a3-43154e513887 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:50.273971 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7a1a8e54-93cb-4ff8-84b9-e74b3756658d admin admin] The token cache is being invalidate because project 70560ff732f0442494762af8e714fe7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:50.274276 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7a1a8e54-93cb-4ff8-84b9-e74b3756658d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:50.274945 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7a1a8e54-93cb-4ff8-84b9-e74b3756658d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:50.276494 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 954/1940] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:50 2022] DELETE /identity/v3/projects/70560ff732f0442494762af8e714fe7e => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:50.282732 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d65a3-a275-4bbb-90fc-0fce3a543732 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:50.283094 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d65a3-a275-4bbb-90fc-0fce3a543732 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:50.283677 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d65a3-a275-4bbb-90fc-0fce3a543732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:50.307177 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4e7d65a3-a275-4bbb-90fc-0fce3a543732 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:50.313900 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4e7d65a3-a275-4bbb-90fc-0fce3a543732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:50.327911 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:50.340955 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d1d3b-428e-4dde-96a6-4d9ab7d5b3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:50.341793 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d1d3b-428e-4dde-96a6-4d9ab7d5b3aa admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:50.342396 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d1d3b-428e-4dde-96a6-4d9ab7d5b3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:50.342742 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d1d3b-428e-4dde-96a6-4d9ab7d5b3aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:50.343503 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601d1d3b-428e-4dde-96a6-4d9ab7d5b3aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:50.345980 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601d1d3b-428e-4dde-96a6-4d9ab7d5b3aa admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:50.385685 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 987/1941] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:50 2022] POST /identity/v3/projects => generated 387 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:09:50.397599 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d1d3b-428e-4dde-96a6-4d9ab7d5b3aa admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:50.402276 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-601d1d3b-428e-4dde-96a6-4d9ab7d5b3aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:50.420559 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 955/1942] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:50.429069 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e749d-f202-4d8d-a6b5-b94c605569fc None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:50.429484 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e749d-f202-4d8d-a6b5-b94c605569fc None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:50.429785 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e749d-f202-4d8d-a6b5-b94c605569fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:50.451287 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-777e749d-f202-4d8d-a6b5-b94c605569fc None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:50.458386 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-777e749d-f202-4d8d-a6b5-b94c605569fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:50.512647 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d1d3b-428e-4dde-96a6-4d9ab7d5b3aa admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:50.518748 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9bf28c-e232-4565-a9b9-415c3cd3abbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:50.519585 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bf28c-e232-4565-a9b9-415c3cd3abbe admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:50.519986 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bf28c-e232-4565-a9b9-415c3cd3abbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:50.520618 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bf28c-e232-4565-a9b9-415c3cd3abbe admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:50.524576 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9bf28c-e232-4565-a9b9-415c3cd3abbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:50.529550 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9bf28c-e232-4565-a9b9-415c3cd3abbe admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:50.563525 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 988/1943] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:50 2022] POST /identity/v3/users => generated 294 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:50.564256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9bf28c-e232-4565-a9b9-415c3cd3abbe admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:50.570394 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 956/1944] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:50.574152 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9bf28c-e232-4565-a9b9-415c3cd3abbe admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:50.574800 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e749d-f202-4d8d-a6b5-b94c605569fc None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:50.578834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d9bfe-c56e-4a35-a6e4-171aae3dd80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:50.579437 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d9bfe-c56e-4a35-a6e4-171aae3dd80b admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:50.579721 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d9bfe-c56e-4a35-a6e4-171aae3dd80b admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:50.580000 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d9bfe-c56e-4a35-a6e4-171aae3dd80b admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:50.580729 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10d9bfe-c56e-4a35-a6e4-171aae3dd80b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:50.582319 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10d9bfe-c56e-4a35-a6e4-171aae3dd80b admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:50.585547 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e749d-f202-4d8d-a6b5-b94c605569fc None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:50.590544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce3f71-3b4e-460f-8e9c-ff7f32775da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:50.591172 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce3f71-3b4e-460f-8e9c-ff7f32775da0 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:50.591525 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce3f71-3b4e-460f-8e9c-ff7f32775da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:50.591878 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce3f71-3b4e-460f-8e9c-ff7f32775da0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:50.593020 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ce3f71-3b4e-460f-8e9c-ff7f32775da0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:50.593316 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 989/1945] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:50.593418 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ce3f71-3b4e-460f-8e9c-ff7f32775da0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:50.595152 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ce3f71-3b4e-460f-8e9c-ff7f32775da0 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:50.597184 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d9bfe-c56e-4a35-a6e4-171aae3dd80b admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:50.608361 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d9bfe-c56e-4a35-a6e4-171aae3dd80b admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:50.609708 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 957/1946] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:09:50.614561 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce3f71-3b4e-460f-8e9c-ff7f32775da0 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:50.617063 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e66dc33-ff13-4f01-90cf-33a8f5d444c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:50.617722 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66dc33-ff13-4f01-90cf-33a8f5d444c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:50.617981 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66dc33-ff13-4f01-90cf-33a8f5d444c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:50.618260 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66dc33-ff13-4f01-90cf-33a8f5d444c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:50.618965 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e66dc33-ff13-4f01-90cf-33a8f5d444c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:50.620602 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e66dc33-ff13-4f01-90cf-33a8f5d444c6 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:50.626268 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce3f71-3b4e-460f-8e9c-ff7f32775da0 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:50.631016 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73a2489-80c3-4221-b49e-fc6581e39fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:50.631640 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a2489-80c3-4221-b49e-fc6581e39fc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:50.631929 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a2489-80c3-4221-b49e-fc6581e39fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:50.632211 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a2489-80c3-4221-b49e-fc6581e39fc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:50.635622 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 990/1947] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:50.635743 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73a2489-80c3-4221-b49e-fc6581e39fc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:50.635743 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73a2489-80c3-4221-b49e-fc6581e39fc7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:50.636619 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e66dc33-ff13-4f01-90cf-33a8f5d444c6 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:50.647316 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e66dc33-ff13-4f01-90cf-33a8f5d444c6 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:50.654364 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aee395-3d09-41d6-a55a-2d9520b4f5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:50.655062 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aee395-3d09-41d6-a55a-2d9520b4f5ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:50.655348 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aee395-3d09-41d6-a55a-2d9520b4f5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:50.655656 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aee395-3d09-41d6-a55a-2d9520b4f5ed admin admin] PATH_INFO: `/v3/projects/2fed15290a1c492aa43ebb20d32c3ce7/users/61dd43ff01b5498d937617c3543cd9ab/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:50.672167 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 958/1948] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:50 2022] POST /identity/v3/projects => generated 359 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:09:50.673838 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aee395-3d09-41d6-a55a-2d9520b4f5ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fed15290a1c492aa43ebb20d32c3ce7, user_id=61dd43ff01b5498d937617c3543cd9ab, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:50.676144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73a2489-80c3-4221-b49e-fc6581e39fc7 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:50.676792 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aee395-3d09-41d6-a55a-2d9520b4f5ed admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:50.691317 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 991/1949] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:50 2022] PUT /identity/v3/projects/2fed15290a1c492aa43ebb20d32c3ce7/users/61dd43ff01b5498d937617c3543cd9ab/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:50.703404 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d876995-ccbc-4ab0-b80c-72e6ddb760d7 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:50.703704 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d876995-ccbc-4ab0-b80c-72e6ddb760d7 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:50.703989 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d876995-ccbc-4ab0-b80c-72e6ddb760d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:50.735306 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-8d876995-ccbc-4ab0-b80c-72e6ddb760d7 None None] MFA Rules not processed for user `61dd43ff01b5498d937617c3543cd9ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:50.736706 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-8d876995-ccbc-4ab0-b80c-72e6ddb760d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:50.749165 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:50.765500 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73a2489-80c3-4221-b49e-fc6581e39fc7 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:50.803622 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:50.825074 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 992/1950] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:50.829170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d876995-ccbc-4ab0-b80c-72e6ddb760d7 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:50.843350 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab1d9b-64c1-4172-917d-9caaccd532a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:50.844303 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab1d9b-64c1-4172-917d-9caaccd532a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:50.844706 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab1d9b-64c1-4172-917d-9caaccd532a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:50.844985 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab1d9b-64c1-4172-917d-9caaccd532a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:50.845916 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fab1d9b-64c1-4172-917d-9caaccd532a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:50.848227 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fab1d9b-64c1-4172-917d-9caaccd532a7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:50.876066 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 959/1951] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:50 2022] POST /identity/v3/users => generated 280 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:09:50.882414 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab1d9b-64c1-4172-917d-9caaccd532a7 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:50.893852 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab1d9b-64c1-4172-917d-9caaccd532a7 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:50.899962 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab0064-fe6a-400d-afc8-0f5dd8e4493f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:50.900670 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab0064-fe6a-400d-afc8-0f5dd8e4493f admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:50.901094 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab0064-fe6a-400d-afc8-0f5dd8e4493f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:50.901373 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab0064-fe6a-400d-afc8-0f5dd8e4493f admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:50.902208 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab0064-fe6a-400d-afc8-0f5dd8e4493f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:50.904303 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab0064-fe6a-400d-afc8-0f5dd8e4493f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:50.914379 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d876995-ccbc-4ab0-b80c-72e6ddb760d7 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:50.916253 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 960/1952] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:50.920237 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab0064-fe6a-400d-afc8-0f5dd8e4493f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:50.922988 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c84034-7340-4195-ba5a-e9825f6ef5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:50.923703 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c84034-7340-4195-ba5a-e9825f6ef5fe admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:50.924093 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c84034-7340-4195-ba5a-e9825f6ef5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:50.924373 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c84034-7340-4195-ba5a-e9825f6ef5fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:50.925081 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c84034-7340-4195-ba5a-e9825f6ef5fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:50.926962 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c84034-7340-4195-ba5a-e9825f6ef5fe admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:50.934042 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab0064-fe6a-400d-afc8-0f5dd8e4493f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:50.939850 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add51a4b-a1af-45a7-99a4-d307ffee046b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:50.940654 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add51a4b-a1af-45a7-99a4-d307ffee046b admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:50.941018 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add51a4b-a1af-45a7-99a4-d307ffee046b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:50.941329 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add51a4b-a1af-45a7-99a4-d307ffee046b admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:50.942165 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add51a4b-a1af-45a7-99a4-d307ffee046b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:50.944837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add51a4b-a1af-45a7-99a4-d307ffee046b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:50.967358 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 961/1953] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:50.971303 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add51a4b-a1af-45a7-99a4-d307ffee046b admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:50.979202 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 993/1954] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:50 2022] POST /identity/v3/projects => generated 385 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:09:50.982880 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add51a4b-a1af-45a7-99a4-d307ffee046b admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:50.983282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c84034-7340-4195-ba5a-e9825f6ef5fe admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:51.057573 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebfc29-0da3-42ca-816d-31da34c30ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:51.059011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebfc29-0da3-42ca-816d-31da34c30ed3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:51.059355 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebfc29-0da3-42ca-816d-31da34c30ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:51.059694 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebfc29-0da3-42ca-816d-31da34c30ed3 admin admin] PATH_INFO: `/v3/projects/9de3632ad8f948a38aff98d421d14a92/users/3d86baed5b1347659e7582e670766348/roles/926b9f36c24847c582c84af904250abc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:51.074710 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c84034-7340-4195-ba5a-e9825f6ef5fe admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:51.080544 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cf6a1a-4dc2-45d6-88f1-45864ada8434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:51.081361 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf6a1a-4dc2-45d6-88f1-45864ada8434 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:51.081539 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf6a1a-4dc2-45d6-88f1-45864ada8434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:51.081769 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf6a1a-4dc2-45d6-88f1-45864ada8434 admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:51.082445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cf6a1a-4dc2-45d6-88f1-45864ada8434 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:51.084037 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ebfc29-0da3-42ca-816d-31da34c30ed3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9de3632ad8f948a38aff98d421d14a92, user_id=3d86baed5b1347659e7582e670766348, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:51.084493 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cf6a1a-4dc2-45d6-88f1-45864ada8434 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:51.087354 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ebfc29-0da3-42ca-816d-31da34c30ed3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:51.108870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 962/1955] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:50 2022] PUT /identity/v3/projects/9de3632ad8f948a38aff98d421d14a92/users/3d86baed5b1347659e7582e670766348/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:51.116832 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 994/1956] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:50 2022] POST /identity/v3/users => generated 299 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:09:51.124296 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebfc29-0da3-42ca-816d-31da34c30ed3 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:51.124471 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b00331-d1fa-4d11-9009-17f23771bc27 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:51.124471 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b00331-d1fa-4d11-9009-17f23771bc27 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:51.125184 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b00331-d1fa-4d11-9009-17f23771bc27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:51.174556 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-16b00331-d1fa-4d11-9009-17f23771bc27 None None] MFA Rules not processed for user `3d86baed5b1347659e7582e670766348`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:51.175831 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-16b00331-d1fa-4d11-9009-17f23771bc27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:51.207157 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:51.232091 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebfc29-0da3-42ca-816d-31da34c30ed3 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:51.237764 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342573ad-d84e-4c65-96ae-0634fe3bf498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:51.239290 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342573ad-d84e-4c65-96ae-0634fe3bf498 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:51.239537 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342573ad-d84e-4c65-96ae-0634fe3bf498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:51.239879 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342573ad-d84e-4c65-96ae-0634fe3bf498 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:51.241264 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342573ad-d84e-4c65-96ae-0634fe3bf498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:51.243501 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342573ad-d84e-4c65-96ae-0634fe3bf498 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:51.276546 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 963/1957] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:51.278466 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342573ad-d84e-4c65-96ae-0634fe3bf498 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:51.321845 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342573ad-d84e-4c65-96ae-0634fe3bf498 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:51.330635 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 995/1958] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:51 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:51.339486 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125daff-fbba-4fd7-8232-8a603cd4cb79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:51.340394 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125daff-fbba-4fd7-8232-8a603cd4cb79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:51.340781 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125daff-fbba-4fd7-8232-8a603cd4cb79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:51.341120 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125daff-fbba-4fd7-8232-8a603cd4cb79 admin admin] PATH_INFO: `/v3/projects/abce40a916b547c193dc9b866479a952/users/b63839d9070241e7884e83b185b19614/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:51.368938 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b00331-d1fa-4d11-9009-17f23771bc27 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:51.372506 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-16b00331-d1fa-4d11-9009-17f23771bc27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:51.374634 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2125daff-fbba-4fd7-8232-8a603cd4cb79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abce40a916b547c193dc9b866479a952, user_id=b63839d9070241e7884e83b185b19614, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:51.378123 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2125daff-fbba-4fd7-8232-8a603cd4cb79 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:51.381492 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:51.402218 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 964/1959] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:51 2022] PUT /identity/v3/projects/abce40a916b547c193dc9b866479a952/users/b63839d9070241e7884e83b185b19614/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:51.406263 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125daff-fbba-4fd7-8232-8a603cd4cb79 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:51.513946 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125daff-fbba-4fd7-8232-8a603cd4cb79 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:51.519717 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec4652-711f-4708-b909-71a167a9b7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:51.520566 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec4652-711f-4708-b909-71a167a9b7e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:51.520854 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec4652-711f-4708-b909-71a167a9b7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:51.521131 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec4652-711f-4708-b909-71a167a9b7e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:51.522376 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ec4652-711f-4708-b909-71a167a9b7e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:51.523991 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ec4652-711f-4708-b909-71a167a9b7e7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:51.526429 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b00331-d1fa-4d11-9009-17f23771bc27 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:51.538967 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 965/1960] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:51.540470 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec4652-711f-4708-b909-71a167a9b7e7 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:51.550778 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec4652-711f-4708-b909-71a167a9b7e7 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:51.556424 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d786be11-70c9-4eae-8b9b-2f81bda5cae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:51.557179 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786be11-70c9-4eae-8b9b-2f81bda5cae7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:51.557541 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786be11-70c9-4eae-8b9b-2f81bda5cae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:51.557882 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786be11-70c9-4eae-8b9b-2f81bda5cae7 admin admin] PATH_INFO: `/v3/projects/abce40a916b547c193dc9b866479a952/users/b63839d9070241e7884e83b185b19614/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:51.560273 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d786be11-70c9-4eae-8b9b-2f81bda5cae7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abce40a916b547c193dc9b866479a952, user_id=b63839d9070241e7884e83b185b19614, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:51.566890 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d786be11-70c9-4eae-8b9b-2f81bda5cae7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:51.589253 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 966/1961] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:51 2022] PUT /identity/v3/projects/abce40a916b547c193dc9b866479a952/users/b63839d9070241e7884e83b185b19614/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:51.597765 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c87061-ee25-4bfe-962a-25b214f14970 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:51.598530 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c87061-ee25-4bfe-962a-25b214f14970 service cinder] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:51.598817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c87061-ee25-4bfe-962a-25b214f14970 service cinder] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:51.599091 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c87061-ee25-4bfe-962a-25b214f14970 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:51.601323 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f7117-34b1-4f7f-9163-25335d88202b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:51.602867 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f7117-34b1-4f7f-9163-25335d88202b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:51.602867 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f7117-34b1-4f7f-9163-25335d88202b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:51.639152 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a97f7117-34b1-4f7f-9163-25335d88202b None None] MFA Rules not processed for user `b63839d9070241e7884e83b185b19614`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:51.641151 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a97f7117-34b1-4f7f-9163-25335d88202b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:51.682758 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c87061-ee25-4bfe-962a-25b214f14970 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:51.684372 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c87061-ee25-4bfe-962a-25b214f14970 service cinder] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:51.697067 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 996/1962] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:51 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:51.715517 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c87061-ee25-4bfe-962a-25b214f14970 service cinder] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:51.726731 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c87061-ee25-4bfe-962a-25b214f14970 service cinder] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:51.736876 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 967/1963] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:51 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:51.745258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f7117-34b1-4f7f-9163-25335d88202b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:51.746790 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a97f7117-34b1-4f7f-9163-25335d88202b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:51.758357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:51.811989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f7117-34b1-4f7f-9163-25335d88202b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:51.816438 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0986bb5c-c1b8-4325-809f-13c0d9e39fa9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:51.817265 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986bb5c-c1b8-4325-809f-13c0d9e39fa9 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:51.817559 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986bb5c-c1b8-4325-809f-13c0d9e39fa9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:51.817844 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986bb5c-c1b8-4325-809f-13c0d9e39fa9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:51.828951 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bf6881-ba53-427c-a628-43637f7c3821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:51.830044 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0986bb5c-c1b8-4325-809f-13c0d9e39fa9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:51.830246 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf6881-ba53-427c-a628-43637f7c3821 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:51.830246 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf6881-ba53-427c-a628-43637f7c3821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:51.830475 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf6881-ba53-427c-a628-43637f7c3821 admin admin] PATH_INFO: `/v3/users/3d86baed5b1347659e7582e670766348` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:51.830943 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0986bb5c-c1b8-4325-809f-13c0d9e39fa9 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:51.831872 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bf6881-ba53-427c-a628-43637f7c3821 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d86baed5b1347659e7582e670766348)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:51.833951 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bf6881-ba53-427c-a628-43637f7c3821 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:51.842864 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 968/1964] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:51 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:51.905327 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-64bf6881-ba53-427c-a628-43637f7c3821 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d86baed5b1347659e7582e670766348'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:51.906530 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-64bf6881-ba53-427c-a628-43637f7c3821 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d86baed5b1347659e7582e670766348'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:51.927405 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-64bf6881-ba53-427c-a628-43637f7c3821 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d86baed5b1347659e7582e670766348'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:51.939576 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-64bf6881-ba53-427c-a628-43637f7c3821 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d86baed5b1347659e7582e670766348'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:51.940414 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-64bf6881-ba53-427c-a628-43637f7c3821 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d86baed5b1347659e7582e670766348'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:51.962893 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 997/1965] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:51 2022] DELETE /identity/v3/users/3d86baed5b1347659e7582e670766348 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:51.967596 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa3485-e09b-4e07-ad7f-a7f3bad55b05 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:51.968109 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa3485-e09b-4e07-ad7f-a7f3bad55b05 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:51.968611 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa3485-e09b-4e07-ad7f-a7f3bad55b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:51.994057 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:52.010669 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-1aaa3485-e09b-4e07-ad7f-a7f3bad55b05 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:52.012003 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1aaa3485-e09b-4e07-ad7f-a7f3bad55b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:52.109157 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 969/1966] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:52.119380 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bf6881-ba53-427c-a628-43637f7c3821 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:52.121067 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-64bf6881-ba53-427c-a628-43637f7c3821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:52.191015 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bf6881-ba53-427c-a628-43637f7c3821 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:52.195184 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60828ce-761d-41ab-a033-147c10d80d8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:52.196142 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60828ce-761d-41ab-a033-147c10d80d8d service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:52.196552 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60828ce-761d-41ab-a033-147c10d80d8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:52.197128 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60828ce-761d-41ab-a033-147c10d80d8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:52.210334 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60828ce-761d-41ab-a033-147c10d80d8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:52.212029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60828ce-761d-41ab-a033-147c10d80d8d service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:52.228886 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 998/1967] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:52.742134 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaa3485-e09b-4e07-ad7f-a7f3bad55b05 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:52.743998 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1aaa3485-e09b-4e07-ad7f-a7f3bad55b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:52.759187 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaa3485-e09b-4e07-ad7f-a7f3bad55b05 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:52.765402 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be36c8d-ecca-4c77-b63e-2e31af10095d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:52.766371 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be36c8d-ecca-4c77-b63e-2e31af10095d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:52.767034 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be36c8d-ecca-4c77-b63e-2e31af10095d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:52.767329 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be36c8d-ecca-4c77-b63e-2e31af10095d admin admin] PATH_INFO: `/v3/users/61dd43ff01b5498d937617c3543cd9ab` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:52.768718 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be36c8d-ecca-4c77-b63e-2e31af10095d admin admin] RBAC: Authorizing `identity:delete_user(user_id=61dd43ff01b5498d937617c3543cd9ab)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:52.771174 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be36c8d-ecca-4c77-b63e-2e31af10095d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:52.810768 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60828ce-761d-41ab-a033-147c10d80d8d service neutron] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:52.812216 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-f60828ce-761d-41ab-a033-147c10d80d8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:52.837653 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60828ce-761d-41ab-a033-147c10d80d8d service neutron] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:52.846915 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a686ba5-3521-492a-a2bd-eebf27dc522c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:52.847638 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a686ba5-3521-492a-a2bd-eebf27dc522c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:52.847867 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a686ba5-3521-492a-a2bd-eebf27dc522c admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:52.848300 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a686ba5-3521-492a-a2bd-eebf27dc522c admin admin] PATH_INFO: `/v3/projects/9de3632ad8f948a38aff98d421d14a92` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:52.850270 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a686ba5-3521-492a-a2bd-eebf27dc522c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9de3632ad8f948a38aff98d421d14a92)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:52.854877 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a686ba5-3521-492a-a2bd-eebf27dc522c admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:52.906539 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0be36c8d-ecca-4c77-b63e-2e31af10095d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61dd43ff01b5498d937617c3543cd9ab'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:52.942650 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0be36c8d-ecca-4c77-b63e-2e31af10095d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61dd43ff01b5498d937617c3543cd9ab'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:52.944756 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0be36c8d-ecca-4c77-b63e-2e31af10095d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61dd43ff01b5498d937617c3543cd9ab'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:52.958109 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0be36c8d-ecca-4c77-b63e-2e31af10095d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61dd43ff01b5498d937617c3543cd9ab'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:52.959888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0be36c8d-ecca-4c77-b63e-2e31af10095d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61dd43ff01b5498d937617c3543cd9ab'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:52.962972 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:52.977174 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-1a686ba5-3521-492a-a2bd-eebf27dc522c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9de3632ad8f948a38aff98d421d14a92', 'request_id': 'req-1a686ba5-3521-492a-a2bd-eebf27dc522c'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:52.983698 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 970/1968] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:52 2022] DELETE /identity/v3/users/61dd43ff01b5498d937617c3543cd9ab => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:52.989144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a77b6-4d4c-4ca0-b67d-155b7a8eed26 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:52.989755 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a77b6-4d4c-4ca0-b67d-155b7a8eed26 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:52.990242 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a77b6-4d4c-4ca0-b67d-155b7a8eed26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:52.999545 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-1a686ba5-3521-492a-a2bd-eebf27dc522c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9de3632ad8f948a38aff98d421d14a92', 'request_id': 'req-1a686ba5-3521-492a-a2bd-eebf27dc522c'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:53.017277 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-1a686ba5-3521-492a-a2bd-eebf27dc522c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9de3632ad8f948a38aff98d421d14a92', 'request_id': 'req-1a686ba5-3521-492a-a2bd-eebf27dc522c'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:53.037937 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b32a77b6-4d4c-4ca0-b67d-155b7a8eed26 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:53.040082 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b32a77b6-4d4c-4ca0-b67d-155b7a8eed26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:53.046134 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-1a686ba5-3521-492a-a2bd-eebf27dc522c admin admin] The token cache is being invalidate because project 9de3632ad8f948a38aff98d421d14a92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:53.047102 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-1a686ba5-3521-492a-a2bd-eebf27dc522c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:53.053948 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-1a686ba5-3521-492a-a2bd-eebf27dc522c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:53.057515 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 999/1969] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:52 2022] DELETE /identity/v3/projects/9de3632ad8f948a38aff98d421d14a92 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:53.065520 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a8283-f549-4e7f-b4e5-07b595be2a6e None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:53.066267 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a8283-f549-4e7f-b4e5-07b595be2a6e None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:53.066267 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a8283-f549-4e7f-b4e5-07b595be2a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:53.131980 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-b81a8283-f549-4e7f-b4e5-07b595be2a6e None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:53.140261 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-b81a8283-f549-4e7f-b4e5-07b595be2a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:53.191685 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 971/1970] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:53.205469 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a77b6-4d4c-4ca0-b67d-155b7a8eed26 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:53.211751 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b32a77b6-4d4c-4ca0-b67d-155b7a8eed26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:53.263725 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:53.282434 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1000/1971] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:53.290933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5934ab5-2faf-49e7-b41d-40eebe0f0c8e None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:53.291308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5934ab5-2faf-49e7-b41d-40eebe0f0c8e None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:53.291503 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5934ab5-2faf-49e7-b41d-40eebe0f0c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:53.336550 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a77b6-4d4c-4ca0-b67d-155b7a8eed26 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:53.336839 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-e5934ab5-2faf-49e7-b41d-40eebe0f0c8e None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:53.339944 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-e5934ab5-2faf-49e7-b41d-40eebe0f0c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:53.340523 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ed756-9cb8-4a62-8c7e-2505ce5d5ca6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:53.341248 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ed756-9cb8-4a62-8c7e-2505ce5d5ca6 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:53.341505 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ed756-9cb8-4a62-8c7e-2505ce5d5ca6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:53.341800 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ed756-9cb8-4a62-8c7e-2505ce5d5ca6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:53.352033 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1001/1972] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:53.353752 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ed756-9cb8-4a62-8c7e-2505ce5d5ca6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:53.356414 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5934ab5-2faf-49e7-b41d-40eebe0f0c8e None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:53.362615 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ed756-9cb8-4a62-8c7e-2505ce5d5ca6 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:53.368256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5934ab5-2faf-49e7-b41d-40eebe0f0c8e None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:53.375837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 972/1973] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:09:53.382114 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec85006-c611-4de2-94ce-4b1706eceb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:53.382888 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec85006-c611-4de2-94ce-4b1706eceb10 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:53.383066 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec85006-c611-4de2-94ce-4b1706eceb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:53.383309 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec85006-c611-4de2-94ce-4b1706eceb10 admin admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:53.384074 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec85006-c611-4de2-94ce-4b1706eceb10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:53.384388 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec85006-c611-4de2-94ce-4b1706eceb10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:53.388324 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec85006-c611-4de2-94ce-4b1706eceb10 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:53.403579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1002/1974] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:09:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:53.412685 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ed756-9cb8-4a62-8c7e-2505ce5d5ca6 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:53.420590 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ed756-9cb8-4a62-8c7e-2505ce5d5ca6 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:53.425907 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7ccae-5ba7-4638-87dd-c8ac19337247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:53.426559 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7ccae-5ba7-4638-87dd-c8ac19337247 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:53.426875 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7ccae-5ba7-4638-87dd-c8ac19337247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:53.427117 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7ccae-5ba7-4638-87dd-c8ac19337247 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:53.428024 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a7ccae-5ba7-4638-87dd-c8ac19337247 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:53.429983 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a7ccae-5ba7-4638-87dd-c8ac19337247 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:53.463123 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 973/1975] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:53 2022] POST /identity/v3/projects => generated 397 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:53.468092 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec85006-c611-4de2-94ce-4b1706eceb10 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:53.484352 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec85006-c611-4de2-94ce-4b1706eceb10 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:53.565950 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7ccae-5ba7-4638-87dd-c8ac19337247 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:53.566763 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e24ea-5237-45e6-a3d0-3b3e08b3bed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:53.567755 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-42a7ccae-5ba7-4638-87dd-c8ac19337247 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:53.568126 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e24ea-5237-45e6-a3d0-3b3e08b3bed4 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:53.568448 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e24ea-5237-45e6-a3d0-3b3e08b3bed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:53.569025 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e24ea-5237-45e6-a3d0-3b3e08b3bed4 admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:53.569713 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e24ea-5237-45e6-a3d0-3b3e08b3bed4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:53.571780 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e24ea-5237-45e6-a3d0-3b3e08b3bed4 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:53.582308 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7ccae-5ba7-4638-87dd-c8ac19337247 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:53.601775 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfc86d8-f497-4ce2-a2c6-909eb050875b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:53.602518 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc86d8-f497-4ce2-a2c6-909eb050875b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:53.602738 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc86d8-f497-4ce2-a2c6-909eb050875b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:53.603024 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc86d8-f497-4ce2-a2c6-909eb050875b admin admin] PATH_INFO: `/v3/users/80a743e783684d55a572c62368ba7023` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:53.603653 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1003/1976] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:53 2022] POST /identity/v3/users => generated 299 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:53.604605 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfc86d8-f497-4ce2-a2c6-909eb050875b admin admin] RBAC: Authorizing `identity:delete_user(user_id=80a743e783684d55a572c62368ba7023)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:53.604712 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e24ea-5237-45e6-a3d0-3b3e08b3bed4 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:53.606781 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfc86d8-f497-4ce2-a2c6-909eb050875b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:53.619278 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e24ea-5237-45e6-a3d0-3b3e08b3bed4 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:53.624223 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62901302-51e0-4cf8-ac68-c82f80a81c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:53.624912 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62901302-51e0-4cf8-ac68-c82f80a81c3c admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:53.625214 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62901302-51e0-4cf8-ac68-c82f80a81c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:53.625475 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62901302-51e0-4cf8-ac68-c82f80a81c3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:53.626179 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62901302-51e0-4cf8-ac68-c82f80a81c3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:53.627844 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62901302-51e0-4cf8-ac68-c82f80a81c3c admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:53.644611 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1004/1977] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:53.646519 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62901302-51e0-4cf8-ac68-c82f80a81c3c admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:53.657233 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62901302-51e0-4cf8-ac68-c82f80a81c3c admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:53.664009 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727fa68-2276-45a5-9bc6-19ae98a4d328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:53.665113 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727fa68-2276-45a5-9bc6-19ae98a4d328 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:53.665486 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727fa68-2276-45a5-9bc6-19ae98a4d328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:53.665775 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727fa68-2276-45a5-9bc6-19ae98a4d328 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:53.666531 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0727fa68-2276-45a5-9bc6-19ae98a4d328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:53.668238 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0727fa68-2276-45a5-9bc6-19ae98a4d328 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:53.670128 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-bcfc86d8-f497-4ce2-a2c6-909eb050875b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80a743e783684d55a572c62368ba7023'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:53.678971 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1005/1978] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:53.679117 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-bcfc86d8-f497-4ce2-a2c6-909eb050875b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80a743e783684d55a572c62368ba7023'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:53.679687 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-bcfc86d8-f497-4ce2-a2c6-909eb050875b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80a743e783684d55a572c62368ba7023'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:53.682936 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727fa68-2276-45a5-9bc6-19ae98a4d328 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:53.684965 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-0727fa68-2276-45a5-9bc6-19ae98a4d328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:53.687930 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-bcfc86d8-f497-4ce2-a2c6-909eb050875b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80a743e783684d55a572c62368ba7023'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:53.688894 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-bcfc86d8-f497-4ce2-a2c6-909eb050875b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80a743e783684d55a572c62368ba7023'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:53.698714 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727fa68-2276-45a5-9bc6-19ae98a4d328 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:53.706086 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e042fe-eaba-4b11-8ed3-9466278aad45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:53.707649 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e042fe-eaba-4b11-8ed3-9466278aad45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:53.708002 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e042fe-eaba-4b11-8ed3-9466278aad45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:53.708857 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e042fe-eaba-4b11-8ed3-9466278aad45 admin admin] PATH_INFO: `/v3/projects/cab1cdbbd9d547bdb453e49dded21a40/users/4be4d02f6efd46858d18e7c27f4c4a24/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:53.720348 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 974/1979] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:53 2022] DELETE /identity/v3/users/80a743e783684d55a572c62368ba7023 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:53.721819 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a13963-07da-4c96-9da4-7a9ad81f1c65 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:53.722128 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a13963-07da-4c96-9da4-7a9ad81f1c65 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:53.722413 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a13963-07da-4c96-9da4-7a9ad81f1c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:53.734264 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e042fe-eaba-4b11-8ed3-9466278aad45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cab1cdbbd9d547bdb453e49dded21a40, user_id=4be4d02f6efd46858d18e7c27f4c4a24, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:53.737941 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e042fe-eaba-4b11-8ed3-9466278aad45 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:53.746791 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:53.747459 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-90a13963-07da-4c96-9da4-7a9ad81f1c65 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:53.749357 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-90a13963-07da-4c96-9da4-7a9ad81f1c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:53.776720 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1006/1980] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:53 2022] PUT /identity/v3/projects/cab1cdbbd9d547bdb453e49dded21a40/users/4be4d02f6efd46858d18e7c27f4c4a24/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:53.786862 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2f878-1003-4ef6-a2f8-360b09243234 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:53.787840 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2f878-1003-4ef6-a2f8-360b09243234 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:53.788185 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2f878-1003-4ef6-a2f8-360b09243234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:53.868401 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-8eb2f878-1003-4ef6-a2f8-360b09243234 None None] MFA Rules not processed for user `4be4d02f6efd46858d18e7c27f4c4a24`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:53.870079 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-8eb2f878-1003-4ef6-a2f8-360b09243234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:53.873491 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 975/1981] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:53.898764 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a13963-07da-4c96-9da4-7a9ad81f1c65 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:53.905278 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-90a13963-07da-4c96-9da4-7a9ad81f1c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:53.934932 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:54.016246 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1007/1982] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:53 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:54.039802 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb2f878-1003-4ef6-a2f8-360b09243234 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:54.043338 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-8eb2f878-1003-4ef6-a2f8-360b09243234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:54.065290 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a13963-07da-4c96-9da4-7a9ad81f1c65 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:54.150448 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83b5b6-d45f-4515-a2be-575a261c365d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:54.151087 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83b5b6-d45f-4515-a2be-575a261c365d service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:54.151370 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83b5b6-d45f-4515-a2be-575a261c365d service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:54.151645 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83b5b6-d45f-4515-a2be-575a261c365d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:54.210863 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb2f878-1003-4ef6-a2f8-360b09243234 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:54.222600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c8709-6ea5-4d2e-a893-a065d69f28b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:54.223376 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c8709-6ea5-4d2e-a893-a065d69f28b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:54.223855 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c8709-6ea5-4d2e-a893-a065d69f28b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:54.224218 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c8709-6ea5-4d2e-a893-a065d69f28b9 admin admin] PATH_INFO: `/v3/projects/2fed15290a1c492aa43ebb20d32c3ce7` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:54.226333 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107c8709-6ea5-4d2e-a893-a065d69f28b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fed15290a1c492aa43ebb20d32c3ce7)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:54.234072 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107c8709-6ea5-4d2e-a893-a065d69f28b9 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:54.259028 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83b5b6-d45f-4515-a2be-575a261c365d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:54.260653 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83b5b6-d45f-4515-a2be-575a261c365d service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:54.272154 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 976/1983] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:54.273045 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83b5b6-d45f-4515-a2be-575a261c365d service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:54.274720 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1a83b5b6-d45f-4515-a2be-575a261c365d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:54.343649 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-107c8709-6ea5-4d2e-a893-a065d69f28b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fed15290a1c492aa43ebb20d32c3ce7', 'request_id': 'req-107c8709-6ea5-4d2e-a893-a065d69f28b9'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:54.348294 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83b5b6-d45f-4515-a2be-575a261c365d service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:54.361418 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-107c8709-6ea5-4d2e-a893-a065d69f28b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fed15290a1c492aa43ebb20d32c3ce7', 'request_id': 'req-107c8709-6ea5-4d2e-a893-a065d69f28b9'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:54.368950 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-107c8709-6ea5-4d2e-a893-a065d69f28b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fed15290a1c492aa43ebb20d32c3ce7', 'request_id': 'req-107c8709-6ea5-4d2e-a893-a065d69f28b9'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:54.387968 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:54.397405 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-107c8709-6ea5-4d2e-a893-a065d69f28b9 admin admin] The token cache is being invalidate because project 2fed15290a1c492aa43ebb20d32c3ce7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:54.397623 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-107c8709-6ea5-4d2e-a893-a065d69f28b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:54.399640 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-107c8709-6ea5-4d2e-a893-a065d69f28b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:54.411034 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1008/1984] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:54 2022] DELETE /identity/v3/projects/2fed15290a1c492aa43ebb20d32c3ce7 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:54.415485 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c8709-6ea5-4d2e-a893-a065d69f28b9 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:54.418235 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-107c8709-6ea5-4d2e-a893-a065d69f28b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:54.433454 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc353f16-f82a-4926-93ac-61164fd0b5d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:54.434589 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc353f16-f82a-4926-93ac-61164fd0b5d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:54.434896 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc353f16-f82a-4926-93ac-61164fd0b5d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:54.435310 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc353f16-f82a-4926-93ac-61164fd0b5d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:54.437329 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cc353f16-f82a-4926-93ac-61164fd0b5d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:54.444742 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:54.534392 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c8709-6ea5-4d2e-a893-a065d69f28b9 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:54.554658 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0598a6-b32d-4ca4-8961-19d4fcab3651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:54.557068 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0598a6-b32d-4ca4-8961-19d4fcab3651 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:54.557194 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0598a6-b32d-4ca4-8961-19d4fcab3651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:54.557541 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0598a6-b32d-4ca4-8961-19d4fcab3651 admin admin] PATH_INFO: `/v3/users/b63839d9070241e7884e83b185b19614` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:54.558955 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0598a6-b32d-4ca4-8961-19d4fcab3651 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b63839d9070241e7884e83b185b19614)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:54.561655 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0598a6-b32d-4ca4-8961-19d4fcab3651 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:54.625963 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc353f16-f82a-4926-93ac-61164fd0b5d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:54.628135 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc353f16-f82a-4926-93ac-61164fd0b5d6 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:54.641763 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 977/1985] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:54 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:54.666581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-bc0598a6-b32d-4ca4-8961-19d4fcab3651 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b63839d9070241e7884e83b185b19614'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:54.667292 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-bc0598a6-b32d-4ca4-8961-19d4fcab3651 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b63839d9070241e7884e83b185b19614'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:54.687155 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-bc0598a6-b32d-4ca4-8961-19d4fcab3651 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b63839d9070241e7884e83b185b19614'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:54.697259 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-bc0598a6-b32d-4ca4-8961-19d4fcab3651 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b63839d9070241e7884e83b185b19614'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:54.698391 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-bc0598a6-b32d-4ca4-8961-19d4fcab3651 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b63839d9070241e7884e83b185b19614'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:54.715913 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1009/1986] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:54 2022] DELETE /identity/v3/users/b63839d9070241e7884e83b185b19614 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:55.323468 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc353f16-f82a-4926-93ac-61164fd0b5d6 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:55.325563 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cc353f16-f82a-4926-93ac-61164fd0b5d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:55.341106 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc353f16-f82a-4926-93ac-61164fd0b5d6 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:55.460862 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2e3da-d899-4e15-b96b-2a16af8ed91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:55.461674 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2e3da-d899-4e15-b96b-2a16af8ed91c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:55.461964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2e3da-d899-4e15-b96b-2a16af8ed91c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:55.462249 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2e3da-d899-4e15-b96b-2a16af8ed91c admin admin] PATH_INFO: `/v3/projects/86608df68045413981929226e1defe1c` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:55.463428 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2e3da-d899-4e15-b96b-2a16af8ed91c admin admin] RBAC: Authorizing `identity:delete_project(project_id=86608df68045413981929226e1defe1c)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:55.466577 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2e3da-d899-4e15-b96b-2a16af8ed91c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:55.467954 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0598a6-b32d-4ca4-8961-19d4fcab3651 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:55.469474 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-bc0598a6-b32d-4ca4-8961-19d4fcab3651 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:55.490533 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0598a6-b32d-4ca4-8961-19d4fcab3651 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:55.495902 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d86bd7-743d-4953-ac84-6b9c24a5823e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:55.496767 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d86bd7-743d-4953-ac84-6b9c24a5823e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:55.497073 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d86bd7-743d-4953-ac84-6b9c24a5823e admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:55.497328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d86bd7-743d-4953-ac84-6b9c24a5823e admin admin] PATH_INFO: `/v3/projects/abce40a916b547c193dc9b866479a952` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:55.498542 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d86bd7-743d-4953-ac84-6b9c24a5823e admin admin] RBAC: Authorizing `identity:delete_project(project_id=abce40a916b547c193dc9b866479a952)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:55.500443 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d86bd7-743d-4953-ac84-6b9c24a5823e admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:55.527006 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-21c2e3da-d899-4e15-b96b-2a16af8ed91c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86608df68045413981929226e1defe1c', 'request_id': 'req-21c2e3da-d899-4e15-b96b-2a16af8ed91c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:55.559599 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-21c2e3da-d899-4e15-b96b-2a16af8ed91c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86608df68045413981929226e1defe1c', 'request_id': 'req-21c2e3da-d899-4e15-b96b-2a16af8ed91c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:55.577597 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-21c2e3da-d899-4e15-b96b-2a16af8ed91c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86608df68045413981929226e1defe1c', 'request_id': 'req-21c2e3da-d899-4e15-b96b-2a16af8ed91c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:55.591857 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-50d86bd7-743d-4953-ac84-6b9c24a5823e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abce40a916b547c193dc9b866479a952', 'request_id': 'req-50d86bd7-743d-4953-ac84-6b9c24a5823e'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:55.597190 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-21c2e3da-d899-4e15-b96b-2a16af8ed91c admin admin] The token cache is being invalidate because project 86608df68045413981929226e1defe1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:55.597496 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-21c2e3da-d899-4e15-b96b-2a16af8ed91c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:55.598664 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-21c2e3da-d899-4e15-b96b-2a16af8ed91c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:55.600173 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-50d86bd7-743d-4953-ac84-6b9c24a5823e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abce40a916b547c193dc9b866479a952', 'request_id': 'req-50d86bd7-743d-4953-ac84-6b9c24a5823e'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:55.600801 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 978/1987] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:55 2022] DELETE /identity/v3/projects/86608df68045413981929226e1defe1c => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:55.604713 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2e3da-d899-4e15-b96b-2a16af8ed91c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:55.606542 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-50d86bd7-743d-4953-ac84-6b9c24a5823e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abce40a916b547c193dc9b866479a952', 'request_id': 'req-50d86bd7-743d-4953-ac84-6b9c24a5823e'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:55.611464 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-21c2e3da-d899-4e15-b96b-2a16af8ed91c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:55.623486 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-50d86bd7-743d-4953-ac84-6b9c24a5823e admin admin] The token cache is being invalidate because project abce40a916b547c193dc9b866479a952 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:55.623785 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-50d86bd7-743d-4953-ac84-6b9c24a5823e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:55.624413 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-50d86bd7-743d-4953-ac84-6b9c24a5823e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:55.626019 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1010/1988] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:55 2022] DELETE /identity/v3/projects/abce40a916b547c193dc9b866479a952 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:55.634667 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d393ac8-3d54-4a7f-9da8-60a413202ffb None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:55.635049 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d393ac8-3d54-4a7f-9da8-60a413202ffb None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:55.635340 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d393ac8-3d54-4a7f-9da8-60a413202ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:55.643878 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:55.656340 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-6d393ac8-3d54-4a7f-9da8-60a413202ffb None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:55.657534 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-6d393ac8-3d54-4a7f-9da8-60a413202ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:55.669648 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:55.689994 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2e3da-d899-4e15-b96b-2a16af8ed91c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:55.720015 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1011/1989] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:55.728107 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc89ee-37c9-4a1c-83ad-c1f115855b7d None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:55.728403 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc89ee-37c9-4a1c-83ad-c1f115855b7d None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:55.728722 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc89ee-37c9-4a1c-83ad-c1f115855b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:55.746480 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c9d14-ed0e-4fa2-9c5a-07473f37e9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:55.747127 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c9d14-ed0e-4fa2-9c5a-07473f37e9d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:55.747409 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c9d14-ed0e-4fa2-9c5a-07473f37e9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:55.747681 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c9d14-ed0e-4fa2-9c5a-07473f37e9d1 admin admin] PATH_INFO: `/v3/users/40c487f7fe5c4c929ae08662280577bb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:55.748878 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c9d14-ed0e-4fa2-9c5a-07473f37e9d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40c487f7fe5c4c929ae08662280577bb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:55.749096 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-4bdc89ee-37c9-4a1c-83ad-c1f115855b7d None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:55.750345 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-4bdc89ee-37c9-4a1c-83ad-c1f115855b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:55.750781 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c9d14-ed0e-4fa2-9c5a-07473f37e9d1 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:55.759321 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1012/1990] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:55.763478 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc89ee-37c9-4a1c-83ad-c1f115855b7d None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:55.773217 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc89ee-37c9-4a1c-83ad-c1f115855b7d None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:55.778132 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caee0a0-18f7-4328-8149-77abf3892e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:55.778783 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caee0a0-18f7-4328-8149-77abf3892e72 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:55.779023 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caee0a0-18f7-4328-8149-77abf3892e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:55.779303 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caee0a0-18f7-4328-8149-77abf3892e72 admin admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:55.780001 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caee0a0-18f7-4328-8149-77abf3892e72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:55.780309 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caee0a0-18f7-4328-8149-77abf3892e72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:55.781933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caee0a0-18f7-4328-8149-77abf3892e72 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:55.793945 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1013/1991] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:09:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:09:55.798264 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caee0a0-18f7-4328-8149-77abf3892e72 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:55.802832 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f94c9d14-ed0e-4fa2-9c5a-07473f37e9d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40c487f7fe5c4c929ae08662280577bb'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:55.808459 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caee0a0-18f7-4328-8149-77abf3892e72 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:55.811236 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f94c9d14-ed0e-4fa2-9c5a-07473f37e9d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40c487f7fe5c4c929ae08662280577bb'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:55.811921 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f94c9d14-ed0e-4fa2-9c5a-07473f37e9d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40c487f7fe5c4c929ae08662280577bb'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:55.813320 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc7ef5-42ab-4c1e-9976-0a11c2c3b1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:55.813985 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc7ef5-42ab-4c1e-9976-0a11c2c3b1ba admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:55.814270 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc7ef5-42ab-4c1e-9976-0a11c2c3b1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:55.814926 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc7ef5-42ab-4c1e-9976-0a11c2c3b1ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:55.815205 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecc7ef5-42ab-4c1e-9976-0a11c2c3b1ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:55.817977 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecc7ef5-42ab-4c1e-9976-0a11c2c3b1ba admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:55.821424 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f94c9d14-ed0e-4fa2-9c5a-07473f37e9d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40c487f7fe5c4c929ae08662280577bb'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:55.821959 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f94c9d14-ed0e-4fa2-9c5a-07473f37e9d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40c487f7fe5c4c929ae08662280577bb'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:55.844220 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 979/1992] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:55 2022] DELETE /identity/v3/users/40c487f7fe5c4c929ae08662280577bb => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:55.850423 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1014/1993] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:55 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:55.851777 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c9d14-ed0e-4fa2-9c5a-07473f37e9d1 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:55.853366 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f94c9d14-ed0e-4fa2-9c5a-07473f37e9d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:55.903362 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:55.921340 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c9d14-ed0e-4fa2-9c5a-07473f37e9d1 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:55.998278 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6482dba-a0fa-4af9-82ee-82a8d47d9f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:55.999309 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6482dba-a0fa-4af9-82ee-82a8d47d9f43 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:55.999662 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6482dba-a0fa-4af9-82ee-82a8d47d9f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:56.000187 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6482dba-a0fa-4af9-82ee-82a8d47d9f43 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:56.001039 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6482dba-a0fa-4af9-82ee-82a8d47d9f43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:56.003907 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6482dba-a0fa-4af9-82ee-82a8d47d9f43 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:56.032160 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 980/1994] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:55 2022] POST /identity/v3/users => generated 287 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:09:56.033588 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc7ef5-42ab-4c1e-9976-0a11c2c3b1ba admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:56.046607 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc7ef5-42ab-4c1e-9976-0a11c2c3b1ba admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:56.051965 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb1961-6365-4758-a826-3861b205db9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:56.052720 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb1961-6365-4758-a826-3861b205db9b admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:56.052987 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb1961-6365-4758-a826-3861b205db9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:56.053299 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb1961-6365-4758-a826-3861b205db9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:56.054153 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafb1961-6365-4758-a826-3861b205db9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:56.055946 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafb1961-6365-4758-a826-3861b205db9b admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:56.067574 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1015/1995] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:56.072313 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6482dba-a0fa-4af9-82ee-82a8d47d9f43 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:56.083289 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6482dba-a0fa-4af9-82ee-82a8d47d9f43 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:56.089247 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe7394-b3f4-4b3b-9c04-71ec8db94075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:56.089877 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe7394-b3f4-4b3b-9c04-71ec8db94075 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:56.090159 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe7394-b3f4-4b3b-9c04-71ec8db94075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:56.090433 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe7394-b3f4-4b3b-9c04-71ec8db94075 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:56.091164 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe7394-b3f4-4b3b-9c04-71ec8db94075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:56.093051 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe7394-b3f4-4b3b-9c04-71ec8db94075 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:56.104586 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 981/1996] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:56.108342 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb1961-6365-4758-a826-3861b205db9b admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:56.122117 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb1961-6365-4758-a826-3861b205db9b admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:56.128597 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702a404-06e5-41d7-a325-f2d5a35c1ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:56.129336 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702a404-06e5-41d7-a325-f2d5a35c1ab9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:56.129614 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702a404-06e5-41d7-a325-f2d5a35c1ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:56.129889 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702a404-06e5-41d7-a325-f2d5a35c1ab9 admin admin] PATH_INFO: `/v3/projects/1e3ef71d88fb469193f0ae827732535e/users/2239be306bfc477aab1c19a44af143e7/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:56.141374 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe7394-b3f4-4b3b-9c04-71ec8db94075 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:56.145190 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-04fe7394-b3f4-4b3b-9c04-71ec8db94075 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:56.154274 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7702a404-06e5-41d7-a325-f2d5a35c1ab9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e3ef71d88fb469193f0ae827732535e, user_id=2239be306bfc477aab1c19a44af143e7, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:56.156646 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7702a404-06e5-41d7-a325-f2d5a35c1ab9 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:56.162866 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe7394-b3f4-4b3b-9c04-71ec8db94075 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:56.168410 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ff0c6-d815-4439-9a85-9b0bd1e5a952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:56.169257 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ff0c6-d815-4439-9a85-9b0bd1e5a952 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:56.169543 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ff0c6-d815-4439-9a85-9b0bd1e5a952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:56.169821 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ff0c6-d815-4439-9a85-9b0bd1e5a952 admin admin] PATH_INFO: `/v3/projects/41482dd8164d47f0bbc52b813efe5e89` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:56.171129 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72ff0c6-d815-4439-9a85-9b0bd1e5a952 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41482dd8164d47f0bbc52b813efe5e89)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:56.172946 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72ff0c6-d815-4439-9a85-9b0bd1e5a952 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:56.175470 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1016/1997] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:56 2022] PUT /identity/v3/projects/1e3ef71d88fb469193f0ae827732535e/users/2239be306bfc477aab1c19a44af143e7/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:56.186809 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8cc4d1-1fa0-4c72-baeb-f6f10b997246 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:56.187204 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8cc4d1-1fa0-4c72-baeb-f6f10b997246 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:56.187569 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8cc4d1-1fa0-4c72-baeb-f6f10b997246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:56.218524 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-dc8cc4d1-1fa0-4c72-baeb-f6f10b997246 None None] MFA Rules not processed for user `2239be306bfc477aab1c19a44af143e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:56.219766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-dc8cc4d1-1fa0-4c72-baeb-f6f10b997246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:56.225323 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e72ff0c6-d815-4439-9a85-9b0bd1e5a952 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41482dd8164d47f0bbc52b813efe5e89', 'request_id': 'req-e72ff0c6-d815-4439-9a85-9b0bd1e5a952'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:56.233172 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e72ff0c6-d815-4439-9a85-9b0bd1e5a952 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41482dd8164d47f0bbc52b813efe5e89', 'request_id': 'req-e72ff0c6-d815-4439-9a85-9b0bd1e5a952'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:56.241066 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e72ff0c6-d815-4439-9a85-9b0bd1e5a952 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41482dd8164d47f0bbc52b813efe5e89', 'request_id': 'req-e72ff0c6-d815-4439-9a85-9b0bd1e5a952'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:56.251393 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e72ff0c6-d815-4439-9a85-9b0bd1e5a952 admin admin] The token cache is being invalidate because project 41482dd8164d47f0bbc52b813efe5e89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:56.251683 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e72ff0c6-d815-4439-9a85-9b0bd1e5a952 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:56.252340 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e72ff0c6-d815-4439-9a85-9b0bd1e5a952 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:56.253911 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 982/1998] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:56 2022] DELETE /identity/v3/projects/41482dd8164d47f0bbc52b813efe5e89 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:56.259372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a4893-43f1-4ce4-ac56-3a85787a2928 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:56.259653 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a4893-43f1-4ce4-ac56-3a85787a2928 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:56.259928 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a4893-43f1-4ce4-ac56-3a85787a2928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:56.278353 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c03a4893-43f1-4ce4-ac56-3a85787a2928 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:56.279551 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c03a4893-43f1-4ce4-ac56-3a85787a2928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:56.291287 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1017/1999] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:56.292317 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8cc4d1-1fa0-4c72-baeb-f6f10b997246 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:56.293669 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-dc8cc4d1-1fa0-4c72-baeb-f6f10b997246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:56.338778 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:56.353538 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:56.391345 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8cc4d1-1fa0-4c72-baeb-f6f10b997246 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:56.391561 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 983/2000] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:56.391561 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d66cd7-b23f-4b4a-abd2-da6c5c66cba9 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:56.391561 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d66cd7-b23f-4b4a-abd2-da6c5c66cba9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:56.391561 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d66cd7-b23f-4b4a-abd2-da6c5c66cba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:56.391826 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147c57c-b9bf-4d40-ba89-87084293e4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:56.391826 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147c57c-b9bf-4d40-ba89-87084293e4d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:56.392135 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147c57c-b9bf-4d40-ba89-87084293e4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:56.392355 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147c57c-b9bf-4d40-ba89-87084293e4d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:56.393105 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3147c57c-b9bf-4d40-ba89-87084293e4d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:56.394923 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3147c57c-b9bf-4d40-ba89-87084293e4d2 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:56.425731 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-89d66cd7-b23f-4b4a-abd2-da6c5c66cba9 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:56.428217 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-89d66cd7-b23f-4b4a-abd2-da6c5c66cba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:56.429741 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1018/2001] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:56 2022] POST /identity/v3/projects => generated 375 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:56.430629 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147c57c-b9bf-4d40-ba89-87084293e4d2 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:56.512053 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 984/2002] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:56.515252 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d66cd7-b23f-4b4a-abd2-da6c5c66cba9 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:56.527088 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147c57c-b9bf-4d40-ba89-87084293e4d2 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:56.528286 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d66cd7-b23f-4b4a-abd2-da6c5c66cba9 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:56.541867 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b527845f-adcd-410b-84da-02288451091e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:56.541973 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973cd86-3886-4db6-a91d-790c1eb95880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:56.542083 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527845f-adcd-410b-84da-02288451091e admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:56.542173 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973cd86-3886-4db6-a91d-790c1eb95880 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:56.542245 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527845f-adcd-410b-84da-02288451091e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:56.542312 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973cd86-3886-4db6-a91d-790c1eb95880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:56.542386 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527845f-adcd-410b-84da-02288451091e admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:56.542455 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973cd86-3886-4db6-a91d-790c1eb95880 admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:56.542531 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b527845f-adcd-410b-84da-02288451091e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:56.542531 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b527845f-adcd-410b-84da-02288451091e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:56.542661 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973cd86-3886-4db6-a91d-790c1eb95880 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:56.542735 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b527845f-adcd-410b-84da-02288451091e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:56.543478 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973cd86-3886-4db6-a91d-790c1eb95880 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:56.557792 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 985/2003] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:09:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:09:56.558835 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b527845f-adcd-410b-84da-02288451091e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:56.573373 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b527845f-adcd-410b-84da-02288451091e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:56.586559 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1019/2004] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:56 2022] POST /identity/v3/users => generated 294 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:56.590729 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973cd86-3886-4db6-a91d-790c1eb95880 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:56.594611 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af7958-281c-4827-ae69-5e2e708bbcc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:56.596264 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af7958-281c-4827-ae69-5e2e708bbcc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:56.596741 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af7958-281c-4827-ae69-5e2e708bbcc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:56.597558 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af7958-281c-4827-ae69-5e2e708bbcc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:56.598830 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44af7958-281c-4827-ae69-5e2e708bbcc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:56.602998 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44af7958-281c-4827-ae69-5e2e708bbcc9 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:56.614046 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973cd86-3886-4db6-a91d-790c1eb95880 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:56.627872 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d090f72c-9900-43bd-be1f-8e5964a49397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:56.630352 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d090f72c-9900-43bd-be1f-8e5964a49397 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:56.631245 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d090f72c-9900-43bd-be1f-8e5964a49397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:56.631606 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d090f72c-9900-43bd-be1f-8e5964a49397 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:56.634308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d090f72c-9900-43bd-be1f-8e5964a49397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:56.637395 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d090f72c-9900-43bd-be1f-8e5964a49397 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:56.648556 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1020/2005] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:56.652136 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d090f72c-9900-43bd-be1f-8e5964a49397 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:56.664704 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d090f72c-9900-43bd-be1f-8e5964a49397 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:56.666669 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 986/2006] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:56 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:56.671628 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af7958-281c-4827-ae69-5e2e708bbcc9 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:56.738918 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02599f93-2dc9-498b-9dd2-c2bf0f2626c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:56.739861 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02599f93-2dc9-498b-9dd2-c2bf0f2626c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:56.740740 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02599f93-2dc9-498b-9dd2-c2bf0f2626c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:56.741237 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02599f93-2dc9-498b-9dd2-c2bf0f2626c5 admin admin] PATH_INFO: `/v3/projects/214c525c2fbc4f768aba9808160cfb54/users/7521d7b2011e4a0cba4966966bf41efa/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:56.764427 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02599f93-2dc9-498b-9dd2-c2bf0f2626c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=214c525c2fbc4f768aba9808160cfb54, user_id=7521d7b2011e4a0cba4966966bf41efa, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:56.767601 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02599f93-2dc9-498b-9dd2-c2bf0f2626c5 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:56.768005 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af7958-281c-4827-ae69-5e2e708bbcc9 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:56.774323 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79722b1-ea4b-46cd-82f8-cc64ab701ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:56.775026 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79722b1-ea4b-46cd-82f8-cc64ab701ac6 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:56.775312 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79722b1-ea4b-46cd-82f8-cc64ab701ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:56.775592 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79722b1-ea4b-46cd-82f8-cc64ab701ac6 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:56.776267 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79722b1-ea4b-46cd-82f8-cc64ab701ac6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:56.778253 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79722b1-ea4b-46cd-82f8-cc64ab701ac6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:56.796519 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1021/2007] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:56 2022] PUT /identity/v3/projects/214c525c2fbc4f768aba9808160cfb54/users/7521d7b2011e4a0cba4966966bf41efa/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:56.800621 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02599f93-2dc9-498b-9dd2-c2bf0f2626c5 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:56.815571 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 987/2008] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:56 2022] POST /identity/v3/users => generated 292 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:09:56.820089 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79722b1-ea4b-46cd-82f8-cc64ab701ac6 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:56.829826 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:56.835037 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79722b1-ea4b-46cd-82f8-cc64ab701ac6 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:56.876590 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02599f93-2dc9-498b-9dd2-c2bf0f2626c5 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:56.881830 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370ed85-a008-4f6a-aaad-0dbbe1818269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:56.882484 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370ed85-a008-4f6a-aaad-0dbbe1818269 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:56.882834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370ed85-a008-4f6a-aaad-0dbbe1818269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:56.883115 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370ed85-a008-4f6a-aaad-0dbbe1818269 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:56.883845 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8370ed85-a008-4f6a-aaad-0dbbe1818269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:56.885473 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8370ed85-a008-4f6a-aaad-0dbbe1818269 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:56.891136 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:56.899048 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1022/2009] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:56.900201 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370ed85-a008-4f6a-aaad-0dbbe1818269 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:56.901916 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2783a7-3d00-4007-9e54-288de8c34044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:56.902560 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2783a7-3d00-4007-9e54-288de8c34044 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:56.903194 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2783a7-3d00-4007-9e54-288de8c34044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:56.903844 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2783a7-3d00-4007-9e54-288de8c34044 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:56.904620 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2783a7-3d00-4007-9e54-288de8c34044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:56.906279 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2783a7-3d00-4007-9e54-288de8c34044 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:56.910300 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370ed85-a008-4f6a-aaad-0dbbe1818269 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:56.915204 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b8a647-8cec-4a30-8cc7-ff23423800a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:56.915891 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b8a647-8cec-4a30-8cc7-ff23423800a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:56.916171 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b8a647-8cec-4a30-8cc7-ff23423800a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:56.916448 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b8a647-8cec-4a30-8cc7-ff23423800a1 admin admin] PATH_INFO: `/v3/projects/214c525c2fbc4f768aba9808160cfb54/users/7521d7b2011e4a0cba4966966bf41efa/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:56.918427 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b8a647-8cec-4a30-8cc7-ff23423800a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=214c525c2fbc4f768aba9808160cfb54, user_id=7521d7b2011e4a0cba4966966bf41efa, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:56.922375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 988/2010] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:56.922501 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b8a647-8cec-4a30-8cc7-ff23423800a1 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:56.923448 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2783a7-3d00-4007-9e54-288de8c34044 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:56.931969 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1023/2011] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:56 2022] PUT /identity/v3/projects/214c525c2fbc4f768aba9808160cfb54/users/7521d7b2011e4a0cba4966966bf41efa/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:56.934674 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2783a7-3d00-4007-9e54-288de8c34044 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:56.942263 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b1408-1cd2-439d-ac44-17a0958bc279 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:56.942557 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b1408-1cd2-439d-ac44-17a0958bc279 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:56.942839 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b1408-1cd2-439d-ac44-17a0958bc279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:56.972236 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-f78b1408-1cd2-439d-ac44-17a0958bc279 None None] MFA Rules not processed for user `7521d7b2011e4a0cba4966966bf41efa`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:56.973572 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-f78b1408-1cd2-439d-ac44-17a0958bc279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:56.995516 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f03de1-02c2-4417-8c47-a4751a106a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:56.996229 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f03de1-02c2-4417-8c47-a4751a106a9a admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:56.996613 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f03de1-02c2-4417-8c47-a4751a106a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:56.996917 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f03de1-02c2-4417-8c47-a4751a106a9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:56.997616 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f03de1-02c2-4417-8c47-a4751a106a9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:56.999236 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f03de1-02c2-4417-8c47-a4751a106a9a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:57.020718 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 989/2012] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:09:57.021996 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f03de1-02c2-4417-8c47-a4751a106a9a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:57.037705 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f03de1-02c2-4417-8c47-a4751a106a9a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:57.037705 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10216b7-23d9-4a83-a9dc-76499fc6fb76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:57.038051 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10216b7-23d9-4a83-a9dc-76499fc6fb76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:57.038338 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10216b7-23d9-4a83-a9dc-76499fc6fb76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:57.038740 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10216b7-23d9-4a83-a9dc-76499fc6fb76 admin admin] PATH_INFO: `/v3/projects/f2d2b6f165c54c12b309c53f025ac721/users/ea46dc35b0e0450e969c87e4851d47dd/roles/926b9f36c24847c582c84af904250abc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:57.054265 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1024/2013] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:56 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:57.057551 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10216b7-23d9-4a83-a9dc-76499fc6fb76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2d2b6f165c54c12b309c53f025ac721, user_id=ea46dc35b0e0450e969c87e4851d47dd, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:57.060256 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10216b7-23d9-4a83-a9dc-76499fc6fb76 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:57.065484 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b1408-1cd2-439d-ac44-17a0958bc279 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:57.066994 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-f78b1408-1cd2-439d-ac44-17a0958bc279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:57.076151 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 990/2014] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:57 2022] PUT /identity/v3/projects/f2d2b6f165c54c12b309c53f025ac721/users/ea46dc35b0e0450e969c87e4851d47dd/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:09:57.088060 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09c684-60f1-42a0-b411-aa8f3bf073cd None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:57.088803 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09c684-60f1-42a0-b411-aa8f3bf073cd None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:57.089114 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09c684-60f1-42a0-b411-aa8f3bf073cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:57.118042 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4f09c684-60f1-42a0-b411-aa8f3bf073cd None None] MFA Rules not processed for user `ea46dc35b0e0450e969c87e4851d47dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:57.119826 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4f09c684-60f1-42a0-b411-aa8f3bf073cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:57.129322 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b1408-1cd2-439d-ac44-17a0958bc279 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:57.171443 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:57.184544 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8e2cf-67a3-4163-ac3b-8e9b62a432c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:57.186794 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8e2cf-67a3-4163-ac3b-8e9b62a432c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:57.186794 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8e2cf-67a3-4163-ac3b-8e9b62a432c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:57.186794 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8e2cf-67a3-4163-ac3b-8e9b62a432c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:57.192104 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 991/2015] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:09:57.204201 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f09c684-60f1-42a0-b411-aa8f3bf073cd None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:57.205651 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4f09c684-60f1-42a0-b411-aa8f3bf073cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:57.218923 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f09c684-60f1-42a0-b411-aa8f3bf073cd None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:57.222585 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d719c19-9eba-4977-a93c-f69ac2f5dc3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:57.223205 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d719c19-9eba-4977-a93c-f69ac2f5dc3d service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:57.223486 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d719c19-9eba-4977-a93c-f69ac2f5dc3d service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:57.223766 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d719c19-9eba-4977-a93c-f69ac2f5dc3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:57.233503 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d719c19-9eba-4977-a93c-f69ac2f5dc3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:57.234968 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d719c19-9eba-4977-a93c-f69ac2f5dc3d service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:57.245837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 992/2016] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:09:57.253722 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df8e2cf-67a3-4163-ac3b-8e9b62a432c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:57.255495 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df8e2cf-67a3-4163-ac3b-8e9b62a432c6 service cinder] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:57.266019 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1025/2017] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:09:57.269753 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d719c19-9eba-4977-a93c-f69ac2f5dc3d service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:57.315123 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:57.344749 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d719c19-9eba-4977-a93c-f69ac2f5dc3d service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:57.349984 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a048e3-cb2d-4729-8675-5a51923026ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:57.350700 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a048e3-cb2d-4729-8675-5a51923026ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:57.351113 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a048e3-cb2d-4729-8675-5a51923026ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:57.351401 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a048e3-cb2d-4729-8675-5a51923026ea admin admin] PATH_INFO: `/v3/users/ea46dc35b0e0450e969c87e4851d47dd` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:57.352735 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a048e3-cb2d-4729-8675-5a51923026ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea46dc35b0e0450e969c87e4851d47dd)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:57.354837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a048e3-cb2d-4729-8675-5a51923026ea admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:57.405843 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a2a048e3-cb2d-4729-8675-5a51923026ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea46dc35b0e0450e969c87e4851d47dd'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:57.414080 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a2a048e3-cb2d-4729-8675-5a51923026ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea46dc35b0e0450e969c87e4851d47dd'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:57.414899 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a2a048e3-cb2d-4729-8675-5a51923026ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea46dc35b0e0450e969c87e4851d47dd'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:57.423005 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a2a048e3-cb2d-4729-8675-5a51923026ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea46dc35b0e0450e969c87e4851d47dd'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:57.424001 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a2a048e3-cb2d-4729-8675-5a51923026ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea46dc35b0e0450e969c87e4851d47dd'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:57.458659 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 993/2018] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:09:57 2022] DELETE /identity/v3/users/ea46dc35b0e0450e969c87e4851d47dd => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:57.463817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febe82b-2f75-4bdd-b775-73095f163891 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:57.464172 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febe82b-2f75-4bdd-b775-73095f163891 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:57.464544 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febe82b-2f75-4bdd-b775-73095f163891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:57.491830 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-6febe82b-2f75-4bdd-b775-73095f163891 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:57.493309 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-6febe82b-2f75-4bdd-b775-73095f163891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:57.569738 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1026/2019] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:57.580022 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a048e3-cb2d-4729-8675-5a51923026ea admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:57.581822 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a2a048e3-cb2d-4729-8675-5a51923026ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:57.687167 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a048e3-cb2d-4729-8675-5a51923026ea admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:57.690844 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103c257-96b7-4b85-a639-2fc62c55ff7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:57.691619 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103c257-96b7-4b85-a639-2fc62c55ff7a service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:57.691921 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103c257-96b7-4b85-a639-2fc62c55ff7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:57.692167 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103c257-96b7-4b85-a639-2fc62c55ff7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:57.705416 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8103c257-96b7-4b85-a639-2fc62c55ff7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:57.707000 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8103c257-96b7-4b85-a639-2fc62c55ff7a service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:57.723794 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 994/2020] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:09:58.182234 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febe82b-2f75-4bdd-b775-73095f163891 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:58.183716 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-6febe82b-2f75-4bdd-b775-73095f163891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:58.197786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febe82b-2f75-4bdd-b775-73095f163891 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:58.203076 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f44bd9-9b46-4ea7-9631-2fe3b84f6424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:58.203772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f44bd9-9b46-4ea7-9631-2fe3b84f6424 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:58.204097 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f44bd9-9b46-4ea7-9631-2fe3b84f6424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:58.204339 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f44bd9-9b46-4ea7-9631-2fe3b84f6424 admin admin] PATH_INFO: `/v3/projects/f2d2b6f165c54c12b309c53f025ac721` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:58.205718 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f44bd9-9b46-4ea7-9631-2fe3b84f6424 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2d2b6f165c54c12b309c53f025ac721)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:58.207800 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f44bd9-9b46-4ea7-9631-2fe3b84f6424 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:58.260113 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-30f44bd9-9b46-4ea7-9631-2fe3b84f6424 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2d2b6f165c54c12b309c53f025ac721', 'request_id': 'req-30f44bd9-9b46-4ea7-9631-2fe3b84f6424'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:58.266621 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-30f44bd9-9b46-4ea7-9631-2fe3b84f6424 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2d2b6f165c54c12b309c53f025ac721', 'request_id': 'req-30f44bd9-9b46-4ea7-9631-2fe3b84f6424'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:58.273356 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-30f44bd9-9b46-4ea7-9631-2fe3b84f6424 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2d2b6f165c54c12b309c53f025ac721', 'request_id': 'req-30f44bd9-9b46-4ea7-9631-2fe3b84f6424'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:58.287531 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-30f44bd9-9b46-4ea7-9631-2fe3b84f6424 admin admin] The token cache is being invalidate because project f2d2b6f165c54c12b309c53f025ac721 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:09:58.287866 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-30f44bd9-9b46-4ea7-9631-2fe3b84f6424 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:58.288664 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-30f44bd9-9b46-4ea7-9631-2fe3b84f6424 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:09:58.290948 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1027/2021] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:09:58 2022] DELETE /identity/v3/projects/f2d2b6f165c54c12b309c53f025ac721 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:58.296087 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be7712-349f-4707-b54a-d8dcb662e8b3 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:58.296377 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be7712-349f-4707-b54a-d8dcb662e8b3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:58.296693 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be7712-349f-4707-b54a-d8dcb662e8b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:58.316247 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-29be7712-349f-4707-b54a-d8dcb662e8b3 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:58.317477 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-29be7712-349f-4707-b54a-d8dcb662e8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:58.368459 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:58.384858 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 995/2022] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:58.387291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a4dec-fef1-473e-848d-8d75342c904e None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:58.387585 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a4dec-fef1-473e-848d-8d75342c904e None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:58.387869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a4dec-fef1-473e-848d-8d75342c904e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:58.408463 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-9d6a4dec-fef1-473e-848d-8d75342c904e None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:58.409708 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-9d6a4dec-fef1-473e-848d-8d75342c904e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:58.417751 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1028/2023] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:58.422080 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29be7712-349f-4707-b54a-d8dcb662e8b3 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:58.432367 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29be7712-349f-4707-b54a-d8dcb662e8b3 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:58.437205 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdaf8b7-af3a-44aa-b66f-459bb95e35ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:58.437831 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdaf8b7-af3a-44aa-b66f-459bb95e35ad admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:58.438117 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdaf8b7-af3a-44aa-b66f-459bb95e35ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:58.438396 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdaf8b7-af3a-44aa-b66f-459bb95e35ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:58.439101 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdaf8b7-af3a-44aa-b66f-459bb95e35ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:09:58.439409 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdaf8b7-af3a-44aa-b66f-459bb95e35ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:58.441049 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdaf8b7-af3a-44aa-b66f-459bb95e35ad admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:58.453476 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 996/2024] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:09:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:09:58.457772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a4dec-fef1-473e-848d-8d75342c904e None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:58.467628 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a4dec-fef1-473e-848d-8d75342c904e None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:58.472206 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c992d909-aced-46c9-9159-68608e2c4153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:58.472859 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992d909-aced-46c9-9159-68608e2c4153 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:58.473151 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992d909-aced-46c9-9159-68608e2c4153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:58.473439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992d909-aced-46c9-9159-68608e2c4153 admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:58.474187 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c992d909-aced-46c9-9159-68608e2c4153 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:58.476102 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c992d909-aced-46c9-9159-68608e2c4153 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:58.487495 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:09:58.508542 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1029/2025] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:09:58 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:58.512246 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdaf8b7-af3a-44aa-b66f-459bb95e35ad admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:58.523199 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdaf8b7-af3a-44aa-b66f-459bb95e35ad admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:58.579659 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacdf158-3297-4f41-8f6c-4dd974ccb9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:58.580325 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdf158-3297-4f41-8f6c-4dd974ccb9a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:58.580643 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdf158-3297-4f41-8f6c-4dd974ccb9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:58.580921 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdf158-3297-4f41-8f6c-4dd974ccb9a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:58.581594 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacdf158-3297-4f41-8f6c-4dd974ccb9a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:58.583532 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacdf158-3297-4f41-8f6c-4dd974ccb9a9 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:58.607846 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 997/2026] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:09:58 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:09:58.612312 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c992d909-aced-46c9-9159-68608e2c4153 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:58.624600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c992d909-aced-46c9-9159-68608e2c4153 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:58.629757 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40727466-fbbe-4c65-85e0-10f16fef499b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:58.631195 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40727466-fbbe-4c65-85e0-10f16fef499b admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:58.631550 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40727466-fbbe-4c65-85e0-10f16fef499b admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:58.631867 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40727466-fbbe-4c65-85e0-10f16fef499b admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:58.632751 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40727466-fbbe-4c65-85e0-10f16fef499b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:58.635093 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40727466-fbbe-4c65-85e0-10f16fef499b admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:58.649286 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1030/2027] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:58.650507 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacdf158-3297-4f41-8f6c-4dd974ccb9a9 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:58.660966 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacdf158-3297-4f41-8f6c-4dd974ccb9a9 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:58.665584 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e406a3e1-33a7-4ac5-a2c9-6a98a8d44363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:58.666199 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406a3e1-33a7-4ac5-a2c9-6a98a8d44363 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:58.666484 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406a3e1-33a7-4ac5-a2c9-6a98a8d44363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:58.666763 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406a3e1-33a7-4ac5-a2c9-6a98a8d44363 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:58.667466 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e406a3e1-33a7-4ac5-a2c9-6a98a8d44363 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:58.669106 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e406a3e1-33a7-4ac5-a2c9-6a98a8d44363 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:58.681699 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 998/2028] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:09:58.685767 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40727466-fbbe-4c65-85e0-10f16fef499b admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:58.696356 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40727466-fbbe-4c65-85e0-10f16fef499b admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:58.701005 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5212ce19-fc94-4623-943b-c522512aa408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:58.701663 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212ce19-fc94-4623-943b-c522512aa408 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:58.701960 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212ce19-fc94-4623-943b-c522512aa408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:58.702225 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212ce19-fc94-4623-943b-c522512aa408 admin admin] PATH_INFO: `/v3/projects/7dfa8cf47ac2427596bc64c20e2224a9/users/624637b656344829ad40d1026e5bd940/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:58.717534 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5212ce19-fc94-4623-943b-c522512aa408 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dfa8cf47ac2427596bc64c20e2224a9, user_id=624637b656344829ad40d1026e5bd940, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:58.720516 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5212ce19-fc94-4623-943b-c522512aa408 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:58.747737 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1031/2029] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:09:58 2022] PUT /identity/v3/projects/7dfa8cf47ac2427596bc64c20e2224a9/users/624637b656344829ad40d1026e5bd940/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:09:58.749941 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a46105-8a60-4261-beb2-287c2251a6c8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:58.750226 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a46105-8a60-4261-beb2-287c2251a6c8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:58.750504 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a46105-8a60-4261-beb2-287c2251a6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:58.790211 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-15a46105-8a60-4261-beb2-287c2251a6c8 None None] MFA Rules not processed for user `624637b656344829ad40d1026e5bd940`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:09:58.791511 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-15a46105-8a60-4261-beb2-287c2251a6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:58.878822 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 999/2030] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:09:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:09:58.892387 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5212ce19-fc94-4623-943b-c522512aa408 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:09:58.894128 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-5212ce19-fc94-4623-943b-c522512aa408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:09:58.954694 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5212ce19-fc94-4623-943b-c522512aa408 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:09:58.959614 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b90e6-0805-4599-a6ff-74f6b7ae9c77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:09:58.960228 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b90e6-0805-4599-a6ff-74f6b7ae9c77 service cinder] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:09:58.960542 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b90e6-0805-4599-a6ff-74f6b7ae9c77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:09:58.960821 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b90e6-0805-4599-a6ff-74f6b7ae9c77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:09:58.969623 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b90e6-0805-4599-a6ff-74f6b7ae9c77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:09:58.971114 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b90e6-0805-4599-a6ff-74f6b7ae9c77 service cinder] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:09:58.979762 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1032/2031] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:09:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:10:07.421041 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a46105-8a60-4261-beb2-287c2251a6c8 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:07.434834 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-15a46105-8a60-4261-beb2-287c2251a6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:07.526661 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:07.551799 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a46105-8a60-4261-beb2-287c2251a6c8 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:07.563286 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b36dc14-8182-4b9b-bd47-623d3b2463fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:07.564042 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36dc14-8182-4b9b-bd47-623d3b2463fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:07.564393 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36dc14-8182-4b9b-bd47-623d3b2463fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:07.564758 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36dc14-8182-4b9b-bd47-623d3b2463fa admin admin] PATH_INFO: `/v3/users/2239be306bfc477aab1c19a44af143e7` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:07.567135 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b36dc14-8182-4b9b-bd47-623d3b2463fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=2239be306bfc477aab1c19a44af143e7)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:07.569299 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b36dc14-8182-4b9b-bd47-623d3b2463fa admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:07.654007 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4b36dc14-8182-4b9b-bd47-623d3b2463fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2239be306bfc477aab1c19a44af143e7'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:07.660874 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4b36dc14-8182-4b9b-bd47-623d3b2463fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2239be306bfc477aab1c19a44af143e7'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:07.663115 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4b36dc14-8182-4b9b-bd47-623d3b2463fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2239be306bfc477aab1c19a44af143e7'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:07.671255 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4b36dc14-8182-4b9b-bd47-623d3b2463fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2239be306bfc477aab1c19a44af143e7'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:07.671936 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4b36dc14-8182-4b9b-bd47-623d3b2463fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2239be306bfc477aab1c19a44af143e7'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:07.692638 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1000/2032] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:10:07 2022] DELETE /identity/v3/users/2239be306bfc477aab1c19a44af143e7 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:07.697111 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a76e3-a0a3-4a0a-a765-438546d8d7be None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:07.697558 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a76e3-a0a3-4a0a-a765-438546d8d7be None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:07.697845 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a76e3-a0a3-4a0a-a765-438546d8d7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:07.733197 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-a19a76e3-a0a3-4a0a-a765-438546d8d7be None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:07.734775 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-a19a76e3-a0a3-4a0a-a765-438546d8d7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:07.810006 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1033/2033] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:10:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:10:07.827188 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b36dc14-8182-4b9b-bd47-623d3b2463fa admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:07.828896 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4b36dc14-8182-4b9b-bd47-623d3b2463fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:07.953690 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b36dc14-8182-4b9b-bd47-623d3b2463fa admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:07.960718 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0e1297-0fbe-4589-84e9-cfb7d77b5c91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:07.961918 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e1297-0fbe-4589-84e9-cfb7d77b5c91 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:07.962399 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e1297-0fbe-4589-84e9-cfb7d77b5c91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:07.962883 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e1297-0fbe-4589-84e9-cfb7d77b5c91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:07.974448 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0e1297-0fbe-4589-84e9-cfb7d77b5c91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:07.975914 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0e1297-0fbe-4589-84e9-cfb7d77b5c91 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:07.991038 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1001/2034] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:10:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:10:08.435098 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19a76e3-a0a3-4a0a-a765-438546d8d7be None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:08.436903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-a19a76e3-a0a3-4a0a-a765-438546d8d7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:08.451309 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19a76e3-a0a3-4a0a-a765-438546d8d7be None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:08.459557 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09353865-9731-4b61-b6c8-438954fd5bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:08.460227 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09353865-9731-4b61-b6c8-438954fd5bf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:08.460523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09353865-9731-4b61-b6c8-438954fd5bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:08.460808 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09353865-9731-4b61-b6c8-438954fd5bf7 admin admin] PATH_INFO: `/v3/projects/1e3ef71d88fb469193f0ae827732535e` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:08.461977 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09353865-9731-4b61-b6c8-438954fd5bf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e3ef71d88fb469193f0ae827732535e)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:08.464209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09353865-9731-4b61-b6c8-438954fd5bf7 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:08.513440 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-09353865-9731-4b61-b6c8-438954fd5bf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e3ef71d88fb469193f0ae827732535e', 'request_id': 'req-09353865-9731-4b61-b6c8-438954fd5bf7'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:08.520173 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-09353865-9731-4b61-b6c8-438954fd5bf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e3ef71d88fb469193f0ae827732535e', 'request_id': 'req-09353865-9731-4b61-b6c8-438954fd5bf7'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:08.521764 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:08.527754 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-09353865-9731-4b61-b6c8-438954fd5bf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e3ef71d88fb469193f0ae827732535e', 'request_id': 'req-09353865-9731-4b61-b6c8-438954fd5bf7'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:08.542233 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-09353865-9731-4b61-b6c8-438954fd5bf7 admin admin] The token cache is being invalidate because project 1e3ef71d88fb469193f0ae827732535e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:10:08.542520 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-09353865-9731-4b61-b6c8-438954fd5bf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:08.543159 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-09353865-9731-4b61-b6c8-438954fd5bf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:08.544751 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1034/2035] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:10:08 2022] DELETE /identity/v3/projects/1e3ef71d88fb469193f0ae827732535e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:08.548877 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0e1297-0fbe-4589-84e9-cfb7d77b5c91 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:08.558984 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0e1297-0fbe-4589-84e9-cfb7d77b5c91 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:08.635959 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b9bb5-1742-4ee5-b6d3-8787b3f3d787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:08.636799 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b9bb5-1742-4ee5-b6d3-8787b3f3d787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:08.637181 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b9bb5-1742-4ee5-b6d3-8787b3f3d787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:08.637559 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b9bb5-1742-4ee5-b6d3-8787b3f3d787 admin admin] PATH_INFO: `/v3/users/7521d7b2011e4a0cba4966966bf41efa` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:08.639137 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5b9bb5-1742-4ee5-b6d3-8787b3f3d787 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7521d7b2011e4a0cba4966966bf41efa)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:08.642084 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5b9bb5-1742-4ee5-b6d3-8787b3f3d787 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:08.661125 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:08.701081 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6c5b9bb5-1742-4ee5-b6d3-8787b3f3d787 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7521d7b2011e4a0cba4966966bf41efa'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:08.711998 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6c5b9bb5-1742-4ee5-b6d3-8787b3f3d787 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7521d7b2011e4a0cba4966966bf41efa'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:08.713403 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6c5b9bb5-1742-4ee5-b6d3-8787b3f3d787 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7521d7b2011e4a0cba4966966bf41efa'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:08.728702 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6c5b9bb5-1742-4ee5-b6d3-8787b3f3d787 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7521d7b2011e4a0cba4966966bf41efa'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:08.729516 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6c5b9bb5-1742-4ee5-b6d3-8787b3f3d787 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7521d7b2011e4a0cba4966966bf41efa'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:08.774697 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1002/2036] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:10:08 2022] DELETE /identity/v3/users/7521d7b2011e4a0cba4966966bf41efa => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:09.179347 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09353865-9731-4b61-b6c8-438954fd5bf7 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:09.181300 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-09353865-9731-4b61-b6c8-438954fd5bf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:09.206548 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09353865-9731-4b61-b6c8-438954fd5bf7 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:09.273177 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f777ead-d5fb-4abd-bf90-17f099250106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:09.273833 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f777ead-d5fb-4abd-bf90-17f099250106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:09.274178 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f777ead-d5fb-4abd-bf90-17f099250106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:09.274449 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f777ead-d5fb-4abd-bf90-17f099250106 admin admin] PATH_INFO: `/v3/projects/214c525c2fbc4f768aba9808160cfb54` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:09.275724 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f777ead-d5fb-4abd-bf90-17f099250106 admin admin] RBAC: Authorizing `identity:delete_project(project_id=214c525c2fbc4f768aba9808160cfb54)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:09.277671 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f777ead-d5fb-4abd-bf90-17f099250106 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:09.330479 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-3f777ead-d5fb-4abd-bf90-17f099250106 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '214c525c2fbc4f768aba9808160cfb54', 'request_id': 'req-3f777ead-d5fb-4abd-bf90-17f099250106'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:09.337436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-3f777ead-d5fb-4abd-bf90-17f099250106 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '214c525c2fbc4f768aba9808160cfb54', 'request_id': 'req-3f777ead-d5fb-4abd-bf90-17f099250106'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:09.344409 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-3f777ead-d5fb-4abd-bf90-17f099250106 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '214c525c2fbc4f768aba9808160cfb54', 'request_id': 'req-3f777ead-d5fb-4abd-bf90-17f099250106'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:09.359141 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-3f777ead-d5fb-4abd-bf90-17f099250106 admin admin] The token cache is being invalidate because project 214c525c2fbc4f768aba9808160cfb54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:10:09.359480 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-3f777ead-d5fb-4abd-bf90-17f099250106 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:09.360080 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-3f777ead-d5fb-4abd-bf90-17f099250106 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:09.361642 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1035/2037] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:10:09 2022] DELETE /identity/v3/projects/214c525c2fbc4f768aba9808160cfb54 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:09.366715 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c097c-035b-4779-b2f7-0e6fbe0fbfe8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:09.367038 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c097c-035b-4779-b2f7-0e6fbe0fbfe8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:09.367325 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c097c-035b-4779-b2f7-0e6fbe0fbfe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:09.391571 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-9f7c097c-035b-4779-b2f7-0e6fbe0fbfe8 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:09.392850 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9f7c097c-035b-4779-b2f7-0e6fbe0fbfe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:09.466784 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1003/2038] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:10:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:09.533833 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d6d3c-c9c5-4632-804e-8838a2dfd3ab None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:09.534146 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d6d3c-c9c5-4632-804e-8838a2dfd3ab None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:09.534430 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d6d3c-c9c5-4632-804e-8838a2dfd3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:09.556407 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-a03d6d3c-c9c5-4632-804e-8838a2dfd3ab None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:09.557678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-a03d6d3c-c9c5-4632-804e-8838a2dfd3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:09.566205 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1036/2039] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:10:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:09.571192 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c097c-035b-4779-b2f7-0e6fbe0fbfe8 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:09.581723 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c097c-035b-4779-b2f7-0e6fbe0fbfe8 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:09.587220 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4183804d-015f-4491-83aa-8412f608d87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:09.588015 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4183804d-015f-4491-83aa-8412f608d87b admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:09.588383 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4183804d-015f-4491-83aa-8412f608d87b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:09.588752 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4183804d-015f-4491-83aa-8412f608d87b admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:09.589476 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4183804d-015f-4491-83aa-8412f608d87b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:10:09.589790 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4183804d-015f-4491-83aa-8412f608d87b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:09.591668 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4183804d-015f-4491-83aa-8412f608d87b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:09.604751 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1004/2040] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:10:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:10:09.609147 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d6d3c-c9c5-4632-804e-8838a2dfd3ab None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:09.620292 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d6d3c-c9c5-4632-804e-8838a2dfd3ab None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:09.627081 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff50b1-cd04-4e9a-a24a-dd729729a097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:09.627711 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff50b1-cd04-4e9a-a24a-dd729729a097 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:09.627993 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff50b1-cd04-4e9a-a24a-dd729729a097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:09.628270 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff50b1-cd04-4e9a-a24a-dd729729a097 admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:09.628988 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff50b1-cd04-4e9a-a24a-dd729729a097 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:09.631520 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff50b1-cd04-4e9a-a24a-dd729729a097 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:09.663114 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1037/2041] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:10:09 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:10:09.664560 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4183804d-015f-4491-83aa-8412f608d87b admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:09.676263 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4183804d-015f-4491-83aa-8412f608d87b admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:09.742530 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e069dc9-1665-4847-976c-a612271b1937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:09.743079 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e069dc9-1665-4847-976c-a612271b1937 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:09.743367 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e069dc9-1665-4847-976c-a612271b1937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:09.743648 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e069dc9-1665-4847-976c-a612271b1937 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:09.744339 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e069dc9-1665-4847-976c-a612271b1937 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:09.746324 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e069dc9-1665-4847-976c-a612271b1937 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:09.772555 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1005/2042] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:10:09 2022] POST /identity/v3/users => generated 291 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:10:09.773921 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff50b1-cd04-4e9a-a24a-dd729729a097 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:09.784749 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff50b1-cd04-4e9a-a24a-dd729729a097 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:09.789499 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ded956-5651-4ef3-ab11-c34e514e2aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:09.790116 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ded956-5651-4ef3-ab11-c34e514e2aec admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:09.790401 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ded956-5651-4ef3-ab11-c34e514e2aec admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:09.790677 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ded956-5651-4ef3-ab11-c34e514e2aec admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:09.791375 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ded956-5651-4ef3-ab11-c34e514e2aec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:09.793010 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ded956-5651-4ef3-ab11-c34e514e2aec admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:09.804682 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1038/2043] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:10:09.808445 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e069dc9-1665-4847-976c-a612271b1937 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:09.824160 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e069dc9-1665-4847-976c-a612271b1937 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:09.828959 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86653c7c-4963-4856-9f80-d280e406fdf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:09.829879 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86653c7c-4963-4856-9f80-d280e406fdf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:09.830377 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86653c7c-4963-4856-9f80-d280e406fdf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:09.830679 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86653c7c-4963-4856-9f80-d280e406fdf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:09.831657 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86653c7c-4963-4856-9f80-d280e406fdf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:09.833418 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86653c7c-4963-4856-9f80-d280e406fdf4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:09.840158 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:09.847948 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1006/2044] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:09 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:10:09.853494 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ded956-5651-4ef3-ab11-c34e514e2aec admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:09.864746 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ded956-5651-4ef3-ab11-c34e514e2aec admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:09.869838 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4761fdf9-cd5e-4abe-b740-f84d4d21ab28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:09.870781 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761fdf9-cd5e-4abe-b740-f84d4d21ab28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:09.870928 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761fdf9-cd5e-4abe-b740-f84d4d21ab28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:09.871126 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761fdf9-cd5e-4abe-b740-f84d4d21ab28 admin admin] PATH_INFO: `/v3/projects/f2afc86f0df14b0f9fb5982da793f79f/users/51acca71ea194e2b9f897e8b72f3b30a/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:09.887271 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4761fdf9-cd5e-4abe-b740-f84d4d21ab28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2afc86f0df14b0f9fb5982da793f79f, user_id=51acca71ea194e2b9f897e8b72f3b30a, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:09.889998 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4761fdf9-cd5e-4abe-b740-f84d4d21ab28 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:09.904155 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1039/2045] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:10:09 2022] PUT /identity/v3/projects/f2afc86f0df14b0f9fb5982da793f79f/users/51acca71ea194e2b9f897e8b72f3b30a/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:09.915144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4225e9-cf12-4e98-94ee-986d0d87160e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:09.915439 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4225e9-cf12-4e98-94ee-986d0d87160e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:09.915724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4225e9-cf12-4e98-94ee-986d0d87160e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:09.946225 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-6e4225e9-cf12-4e98-94ee-986d0d87160e None None] MFA Rules not processed for user `51acca71ea194e2b9f897e8b72f3b30a`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:09.947484 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6e4225e9-cf12-4e98-94ee-986d0d87160e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:10.042530 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1007/2046] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:10:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:10.054403 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4761fdf9-cd5e-4abe-b740-f84d4d21ab28 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:10.056712 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-4761fdf9-cd5e-4abe-b740-f84d4d21ab28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:10.067736 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:10.118386 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4761fdf9-cd5e-4abe-b740-f84d4d21ab28 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:10.122168 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bc1881-4208-40b5-823b-4d2aba41063a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:10.122794 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc1881-4208-40b5-823b-4d2aba41063a service cinder] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:10.123081 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc1881-4208-40b5-823b-4d2aba41063a service cinder] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:10.123360 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc1881-4208-40b5-823b-4d2aba41063a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:10.132924 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bc1881-4208-40b5-823b-4d2aba41063a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:10.134364 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bc1881-4208-40b5-823b-4d2aba41063a service cinder] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:10.144348 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1040/2047] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:10:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:10:14.687761 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4225e9-cf12-4e98-94ee-986d0d87160e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:14.689207 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6e4225e9-cf12-4e98-94ee-986d0d87160e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:14.759147 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4225e9-cf12-4e98-94ee-986d0d87160e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:14.763760 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b625332-5611-4335-969e-232ed1b3c425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:14.764421 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b625332-5611-4335-969e-232ed1b3c425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:14.764726 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b625332-5611-4335-969e-232ed1b3c425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:14.765011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b625332-5611-4335-969e-232ed1b3c425 admin admin] PATH_INFO: `/v3/users/4be4d02f6efd46858d18e7c27f4c4a24` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:14.766199 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b625332-5611-4335-969e-232ed1b3c425 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4be4d02f6efd46858d18e7c27f4c4a24)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:14.768131 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b625332-5611-4335-969e-232ed1b3c425 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:14.841416 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3b625332-5611-4335-969e-232ed1b3c425 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4be4d02f6efd46858d18e7c27f4c4a24'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:14.859826 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3b625332-5611-4335-969e-232ed1b3c425 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4be4d02f6efd46858d18e7c27f4c4a24'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:14.860751 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3b625332-5611-4335-969e-232ed1b3c425 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4be4d02f6efd46858d18e7c27f4c4a24'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:14.874249 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3b625332-5611-4335-969e-232ed1b3c425 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4be4d02f6efd46858d18e7c27f4c4a24'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:14.875160 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3b625332-5611-4335-969e-232ed1b3c425 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4be4d02f6efd46858d18e7c27f4c4a24'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:14.904414 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1008/2048] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:10:14 2022] DELETE /identity/v3/users/4be4d02f6efd46858d18e7c27f4c4a24 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:14.909016 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087aa05-1bba-4f2b-bb43-2bed5a72cfcb None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:14.909463 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087aa05-1bba-4f2b-bb43-2bed5a72cfcb None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:14.909600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087aa05-1bba-4f2b-bb43-2bed5a72cfcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:14.931411 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-2087aa05-1bba-4f2b-bb43-2bed5a72cfcb None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:14.933299 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-2087aa05-1bba-4f2b-bb43-2bed5a72cfcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:15.041897 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1041/2049] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:10:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:15.053032 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b625332-5611-4335-969e-232ed1b3c425 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:15.055848 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3b625332-5611-4335-969e-232ed1b3c425 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:15.068391 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:15.130320 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b625332-5611-4335-969e-232ed1b3c425 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:15.134397 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ede33-d249-4aab-b19b-dcf3479eeacf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:15.135160 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ede33-d249-4aab-b19b-dcf3479eeacf service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:15.135436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ede33-d249-4aab-b19b-dcf3479eeacf service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:15.135709 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ede33-d249-4aab-b19b-dcf3479eeacf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:15.145439 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81ede33-d249-4aab-b19b-dcf3479eeacf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:15.147777 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81ede33-d249-4aab-b19b-dcf3479eeacf service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:15.159067 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1009/2050] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:10:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:10:15.464185 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087aa05-1bba-4f2b-bb43-2bed5a72cfcb None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:15.465690 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-2087aa05-1bba-4f2b-bb43-2bed5a72cfcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:15.479298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087aa05-1bba-4f2b-bb43-2bed5a72cfcb None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:15.484443 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62794fb-9c64-423e-84d7-6d52ec61305c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:15.485691 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62794fb-9c64-423e-84d7-6d52ec61305c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:15.485691 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62794fb-9c64-423e-84d7-6d52ec61305c admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:15.485691 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62794fb-9c64-423e-84d7-6d52ec61305c admin admin] PATH_INFO: `/v3/projects/cab1cdbbd9d547bdb453e49dded21a40` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:15.486848 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62794fb-9c64-423e-84d7-6d52ec61305c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cab1cdbbd9d547bdb453e49dded21a40)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:15.488870 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62794fb-9c64-423e-84d7-6d52ec61305c admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:15.550035 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-b62794fb-9c64-423e-84d7-6d52ec61305c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cab1cdbbd9d547bdb453e49dded21a40', 'request_id': 'req-b62794fb-9c64-423e-84d7-6d52ec61305c'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:15.558956 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-b62794fb-9c64-423e-84d7-6d52ec61305c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cab1cdbbd9d547bdb453e49dded21a40', 'request_id': 'req-b62794fb-9c64-423e-84d7-6d52ec61305c'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:15.567527 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-b62794fb-9c64-423e-84d7-6d52ec61305c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cab1cdbbd9d547bdb453e49dded21a40', 'request_id': 'req-b62794fb-9c64-423e-84d7-6d52ec61305c'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:15.588014 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-b62794fb-9c64-423e-84d7-6d52ec61305c admin admin] The token cache is being invalidate because project cab1cdbbd9d547bdb453e49dded21a40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:10:15.588459 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-b62794fb-9c64-423e-84d7-6d52ec61305c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:15.589129 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-b62794fb-9c64-423e-84d7-6d52ec61305c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:15.590688 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1042/2051] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:10:15 2022] DELETE /identity/v3/projects/cab1cdbbd9d547bdb453e49dded21a40 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:15.943241 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ede33-d249-4aab-b19b-dcf3479eeacf service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:15.945698 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f81ede33-d249-4aab-b19b-dcf3479eeacf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:15.964149 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ede33-d249-4aab-b19b-dcf3479eeacf service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:16.061097 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be7eff-0419-4a02-843d-fb6db98ef33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:16.061763 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be7eff-0419-4a02-843d-fb6db98ef33d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:16.062121 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be7eff-0419-4a02-843d-fb6db98ef33d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:16.062404 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be7eff-0419-4a02-843d-fb6db98ef33d admin admin] PATH_INFO: `/v3/users/51acca71ea194e2b9f897e8b72f3b30a` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:16.063789 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be7eff-0419-4a02-843d-fb6db98ef33d admin admin] RBAC: Authorizing `identity:delete_user(user_id=51acca71ea194e2b9f897e8b72f3b30a)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:16.065865 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be7eff-0419-4a02-843d-fb6db98ef33d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:16.133776 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a3be7eff-0419-4a02-843d-fb6db98ef33d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51acca71ea194e2b9f897e8b72f3b30a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:16.142461 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a3be7eff-0419-4a02-843d-fb6db98ef33d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51acca71ea194e2b9f897e8b72f3b30a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:16.143192 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a3be7eff-0419-4a02-843d-fb6db98ef33d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51acca71ea194e2b9f897e8b72f3b30a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:16.156851 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a3be7eff-0419-4a02-843d-fb6db98ef33d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51acca71ea194e2b9f897e8b72f3b30a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:16.157512 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a3be7eff-0419-4a02-843d-fb6db98ef33d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51acca71ea194e2b9f897e8b72f3b30a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:16.177072 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1010/2052] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:10:15 2022] DELETE /identity/v3/users/51acca71ea194e2b9f897e8b72f3b30a => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:16.181640 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33a5a5-a8c0-4e07-8ebe-382fedfde7af None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:16.182102 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33a5a5-a8c0-4e07-8ebe-382fedfde7af None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:16.182386 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33a5a5-a8c0-4e07-8ebe-382fedfde7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:16.203320 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-4a33a5a5-a8c0-4e07-8ebe-382fedfde7af None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:16.205022 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-4a33a5a5-a8c0-4e07-8ebe-382fedfde7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:16.240363 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:16.273040 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1043/2053] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:10:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:16.283182 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be7eff-0419-4a02-843d-fb6db98ef33d admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:16.284955 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a3be7eff-0419-4a02-843d-fb6db98ef33d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:16.323528 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:16.358048 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be7eff-0419-4a02-843d-fb6db98ef33d admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:16.361872 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a854f7-f481-40b9-9dc8-8133bb5012b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:16.362397 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a854f7-f481-40b9-9dc8-8133bb5012b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:16.362690 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a854f7-f481-40b9-9dc8-8133bb5012b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:16.363343 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a854f7-f481-40b9-9dc8-8133bb5012b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:16.374829 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a854f7-f481-40b9-9dc8-8133bb5012b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:16.376273 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a854f7-f481-40b9-9dc8-8133bb5012b2 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:16.394560 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1011/2054] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:10:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:10:16.430014 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33a5a5-a8c0-4e07-8ebe-382fedfde7af None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:16.431457 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-4a33a5a5-a8c0-4e07-8ebe-382fedfde7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:16.446659 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33a5a5-a8c0-4e07-8ebe-382fedfde7af None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:16.452843 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe1640-52c1-47c7-ba4f-9b3d2ef52a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:16.453743 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe1640-52c1-47c7-ba4f-9b3d2ef52a44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:16.454179 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe1640-52c1-47c7-ba4f-9b3d2ef52a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:16.454737 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe1640-52c1-47c7-ba4f-9b3d2ef52a44 admin admin] PATH_INFO: `/v3/users/4e5b738a1ef6499cb2c9447c80b007a2` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:16.456093 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fe1640-52c1-47c7-ba4f-9b3d2ef52a44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e5b738a1ef6499cb2c9447c80b007a2)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:16.458757 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fe1640-52c1-47c7-ba4f-9b3d2ef52a44 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:16.520881 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-41fe1640-52c1-47c7-ba4f-9b3d2ef52a44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e5b738a1ef6499cb2c9447c80b007a2'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:16.521910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-41fe1640-52c1-47c7-ba4f-9b3d2ef52a44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e5b738a1ef6499cb2c9447c80b007a2'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:16.535861 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-41fe1640-52c1-47c7-ba4f-9b3d2ef52a44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e5b738a1ef6499cb2c9447c80b007a2'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:16.545218 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-41fe1640-52c1-47c7-ba4f-9b3d2ef52a44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e5b738a1ef6499cb2c9447c80b007a2'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:16.546191 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-41fe1640-52c1-47c7-ba4f-9b3d2ef52a44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e5b738a1ef6499cb2c9447c80b007a2'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:16.568222 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1044/2055] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:10:16 2022] DELETE /identity/v3/users/4e5b738a1ef6499cb2c9447c80b007a2 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:16.573368 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050d484-0231-4880-b534-4c33eff75cd7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:16.573779 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050d484-0231-4880-b534-4c33eff75cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:16.574180 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050d484-0231-4880-b534-4c33eff75cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:16.602561 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-2050d484-0231-4880-b534-4c33eff75cd7 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:16.604399 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2050d484-0231-4880-b534-4c33eff75cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:16.679049 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1012/2056] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:10:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:10:16.686466 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe1640-52c1-47c7-ba4f-9b3d2ef52a44 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:16.688027 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-41fe1640-52c1-47c7-ba4f-9b3d2ef52a44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:16.694329 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2050d484-0231-4880-b534-4c33eff75cd7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:16.695940 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2050d484-0231-4880-b534-4c33eff75cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:16.706404 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe1640-52c1-47c7-ba4f-9b3d2ef52a44 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:16.714714 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d482c023-b50c-4183-9819-1352bcb0188e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:16.716231 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d482c023-b50c-4183-9819-1352bcb0188e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:16.716556 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d482c023-b50c-4183-9819-1352bcb0188e admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:16.717231 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d482c023-b50c-4183-9819-1352bcb0188e admin admin] PATH_INFO: `/v3/projects/f2afc86f0df14b0f9fb5982da793f79f` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:16.718080 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d482c023-b50c-4183-9819-1352bcb0188e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2afc86f0df14b0f9fb5982da793f79f)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:16.720266 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d482c023-b50c-4183-9819-1352bcb0188e admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:16.776463 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2050d484-0231-4880-b534-4c33eff75cd7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:16.785247 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d482c023-b50c-4183-9819-1352bcb0188e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2afc86f0df14b0f9fb5982da793f79f', 'request_id': 'req-d482c023-b50c-4183-9819-1352bcb0188e'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:16.793860 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d482c023-b50c-4183-9819-1352bcb0188e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2afc86f0df14b0f9fb5982da793f79f', 'request_id': 'req-d482c023-b50c-4183-9819-1352bcb0188e'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:16.804732 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d482c023-b50c-4183-9819-1352bcb0188e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2afc86f0df14b0f9fb5982da793f79f', 'request_id': 'req-d482c023-b50c-4183-9819-1352bcb0188e'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:16.830907 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d482c023-b50c-4183-9819-1352bcb0188e admin admin] The token cache is being invalidate because project f2afc86f0df14b0f9fb5982da793f79f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:10:16.831051 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d482c023-b50c-4183-9819-1352bcb0188e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:16.831832 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d482c023-b50c-4183-9819-1352bcb0188e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:16.837313 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1045/2057] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:10:16 2022] DELETE /identity/v3/projects/f2afc86f0df14b0f9fb5982da793f79f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:16.838861 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0a100-a5ef-419c-969b-221246f71a3e None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:16.840620 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0a100-a5ef-419c-969b-221246f71a3e None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:16.840762 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0a100-a5ef-419c-969b-221246f71a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:16.866304 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-6ad0a100-a5ef-419c-969b-221246f71a3e None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:16.867783 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-6ad0a100-a5ef-419c-969b-221246f71a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:16.869647 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dfd4a2-96e9-46b9-aa49-45efbe56670f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:16.870283 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dfd4a2-96e9-46b9-aa49-45efbe56670f service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:16.870585 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dfd4a2-96e9-46b9-aa49-45efbe56670f service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:16.870960 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dfd4a2-96e9-46b9-aa49-45efbe56670f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:16.873134 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a5dfd4a2-96e9-46b9-aa49-45efbe56670f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:16.899450 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:16.942705 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:16.958611 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1046/2058] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:10:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:16.964417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dfd4a2-96e9-46b9-aa49-45efbe56670f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:16.966220 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dfd4a2-96e9-46b9-aa49-45efbe56670f service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:16.966968 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a2506-dad1-42ea-bd96-87d8661ddae5 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:16.967265 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a2506-dad1-42ea-bd96-87d8661ddae5 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:16.967552 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a2506-dad1-42ea-bd96-87d8661ddae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:16.976955 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1013/2059] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:10:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:10:16.988649 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-5c6a2506-dad1-42ea-bd96-87d8661ddae5 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:16.990587 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-5c6a2506-dad1-42ea-bd96-87d8661ddae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:17.002426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1047/2060] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:10:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:17.010510 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dfd4a2-96e9-46b9-aa49-45efbe56670f service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:17.027086 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dfd4a2-96e9-46b9-aa49-45efbe56670f service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:17.028104 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a2506-dad1-42ea-bd96-87d8661ddae5 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:17.029709 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-5c6a2506-dad1-42ea-bd96-87d8661ddae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:17.035571 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd67b366-b3d9-4f79-b040-a39b81f0be29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:17.037331 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67b366-b3d9-4f79-b040-a39b81f0be29 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:17.037631 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67b366-b3d9-4f79-b040-a39b81f0be29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:17.037916 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67b366-b3d9-4f79-b040-a39b81f0be29 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:17.038956 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd67b366-b3d9-4f79-b040-a39b81f0be29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:10:17.039329 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd67b366-b3d9-4f79-b040-a39b81f0be29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:17.041779 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd67b366-b3d9-4f79-b040-a39b81f0be29 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:17.046901 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a2506-dad1-42ea-bd96-87d8661ddae5 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:17.054194 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b6633-4c46-40b8-8203-aef6b9349a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:17.054933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b6633-4c46-40b8-8203-aef6b9349a2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:17.055422 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b6633-4c46-40b8-8203-aef6b9349a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:17.055759 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b6633-4c46-40b8-8203-aef6b9349a2e admin admin] PATH_INFO: `/v3/users/624637b656344829ad40d1026e5bd940` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:17.057069 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b6633-4c46-40b8-8203-aef6b9349a2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=624637b656344829ad40d1026e5bd940)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:17.060517 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b6633-4c46-40b8-8203-aef6b9349a2e admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:17.061646 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1014/2061] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:10:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:10:17.062939 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd67b366-b3d9-4f79-b040-a39b81f0be29 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:17.077147 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd67b366-b3d9-4f79-b040-a39b81f0be29 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:17.087905 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea955a5-463b-484d-8fcf-a441600adc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:17.089345 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea955a5-463b-484d-8fcf-a441600adc66 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:17.089824 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea955a5-463b-484d-8fcf-a441600adc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:17.090348 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea955a5-463b-484d-8fcf-a441600adc66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:17.092149 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea955a5-463b-484d-8fcf-a441600adc66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:17.095801 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea955a5-463b-484d-8fcf-a441600adc66 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:17.143549 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1015/2062] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:10:17 2022] POST /identity/v3/projects => generated 385 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:10:17.148518 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea955a5-463b-484d-8fcf-a441600adc66 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:17.164247 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-3d0b6633-4c46-40b8-8203-aef6b9349a2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '624637b656344829ad40d1026e5bd940'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:17.164247 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-3d0b6633-4c46-40b8-8203-aef6b9349a2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '624637b656344829ad40d1026e5bd940'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:17.183738 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-3d0b6633-4c46-40b8-8203-aef6b9349a2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '624637b656344829ad40d1026e5bd940'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:17.194874 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-3d0b6633-4c46-40b8-8203-aef6b9349a2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '624637b656344829ad40d1026e5bd940'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:17.196405 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-3d0b6633-4c46-40b8-8203-aef6b9349a2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '624637b656344829ad40d1026e5bd940'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:17.217785 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1048/2063] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:10:17 2022] DELETE /identity/v3/users/624637b656344829ad40d1026e5bd940 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:17.225535 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadfe6b-2cbf-454f-9597-64331c896ad1 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:17.226402 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadfe6b-2cbf-454f-9597-64331c896ad1 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:17.227087 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadfe6b-2cbf-454f-9597-64331c896ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:17.249782 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea955a5-463b-484d-8fcf-a441600adc66 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:17.262030 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-5aadfe6b-2cbf-454f-9597-64331c896ad1 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:17.264353 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-5aadfe6b-2cbf-454f-9597-64331c896ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:17.357903 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c728f1-34ec-47ac-ac32-17d3c9950ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:17.359075 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c728f1-34ec-47ac-ac32-17d3c9950ec3 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:17.359545 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c728f1-34ec-47ac-ac32-17d3c9950ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:17.360063 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c728f1-34ec-47ac-ac32-17d3c9950ec3 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:17.361023 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c728f1-34ec-47ac-ac32-17d3c9950ec3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:17.362152 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1049/2064] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:10:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:17.370453 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c728f1-34ec-47ac-ac32-17d3c9950ec3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:17.387546 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aadfe6b-2cbf-454f-9597-64331c896ad1 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:17.390590 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-5aadfe6b-2cbf-454f-9597-64331c896ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:17.415587 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1016/2065] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:10:17 2022] POST /identity/v3/users => generated 293 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:10:17.419827 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c728f1-34ec-47ac-ac32-17d3c9950ec3 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:17.424930 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-86c728f1-34ec-47ac-ac32-17d3c9950ec3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:17.460314 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c728f1-34ec-47ac-ac32-17d3c9950ec3 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:17.475252 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c4deac-0c7f-4f9f-ab34-1c226aca2521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:17.476302 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4deac-0c7f-4f9f-ab34-1c226aca2521 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:17.478295 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4deac-0c7f-4f9f-ab34-1c226aca2521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:17.478685 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4deac-0c7f-4f9f-ab34-1c226aca2521 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:17.479511 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c4deac-0c7f-4f9f-ab34-1c226aca2521 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:17.483439 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c4deac-0c7f-4f9f-ab34-1c226aca2521 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:17.508821 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:17.510273 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1017/2066] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:10:17.514614 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c4deac-0c7f-4f9f-ab34-1c226aca2521 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:17.535046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c4deac-0c7f-4f9f-ab34-1c226aca2521 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:17.547665 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e050349-c944-4951-a730-a880589772e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:17.548393 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e050349-c944-4951-a730-a880589772e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:17.548777 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e050349-c944-4951-a730-a880589772e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:17.549132 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e050349-c944-4951-a730-a880589772e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:17.549612 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aadfe6b-2cbf-454f-9597-64331c896ad1 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:17.549976 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e050349-c944-4951-a730-a880589772e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:17.551853 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e050349-c944-4951-a730-a880589772e8 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:17.557233 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e191df-0e0c-4372-b026-6da4b7aa0f2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:17.558352 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e191df-0e0c-4372-b026-6da4b7aa0f2a service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:17.558715 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e191df-0e0c-4372-b026-6da4b7aa0f2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:17.559065 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e191df-0e0c-4372-b026-6da4b7aa0f2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:17.585556 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e191df-0e0c-4372-b026-6da4b7aa0f2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:17.587331 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e191df-0e0c-4372-b026-6da4b7aa0f2a service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:17.597827 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1018/2067] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:10:17.599651 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e050349-c944-4951-a730-a880589772e8 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:17.599892 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1050/2068] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:10:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:10:17.601923 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1e050349-c944-4951-a730-a880589772e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:17.602759 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e191df-0e0c-4372-b026-6da4b7aa0f2a service neutron] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:17.619346 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e050349-c944-4951-a730-a880589772e8 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:17.623535 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e191df-0e0c-4372-b026-6da4b7aa0f2a service neutron] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:17.625111 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82193347-8f1d-466b-aacb-b47456b83409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:17.626099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82193347-8f1d-466b-aacb-b47456b83409 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:17.626572 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82193347-8f1d-466b-aacb-b47456b83409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:17.626993 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82193347-8f1d-466b-aacb-b47456b83409 admin admin] PATH_INFO: `/v3/projects/5324c3957e004c548e5fb243d6a23aaf` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:17.628668 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82193347-8f1d-466b-aacb-b47456b83409 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5324c3957e004c548e5fb243d6a23aaf)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:17.631708 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82193347-8f1d-466b-aacb-b47456b83409 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:17.637887 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22466462-f1b5-475d-b21d-458474d5aeb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:17.638762 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22466462-f1b5-475d-b21d-458474d5aeb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:17.639309 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22466462-f1b5-475d-b21d-458474d5aeb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:17.639785 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22466462-f1b5-475d-b21d-458474d5aeb4 admin admin] PATH_INFO: `/v3/projects/a8630c7be7ea49d5b6d3ef6d46fd653e/users/b6ed2d0b015d4fe39d780306d04fba4d/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:17.664593 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:17.676827 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22466462-f1b5-475d-b21d-458474d5aeb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8630c7be7ea49d5b6d3ef6d46fd653e, user_id=b6ed2d0b015d4fe39d780306d04fba4d, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:17.679853 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22466462-f1b5-475d-b21d-458474d5aeb4 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:17.702166 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1051/2069] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:10:17 2022] PUT /identity/v3/projects/a8630c7be7ea49d5b6d3ef6d46fd653e/users/b6ed2d0b015d4fe39d780306d04fba4d/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:17.707117 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5c018-91bf-439a-888f-49b2f4ef0f94 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:17.707722 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5c018-91bf-439a-888f-49b2f4ef0f94 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:17.708882 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5c018-91bf-439a-888f-49b2f4ef0f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:17.737303 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-82193347-8f1d-466b-aacb-b47456b83409 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5324c3957e004c548e5fb243d6a23aaf', 'request_id': 'req-82193347-8f1d-466b-aacb-b47456b83409'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:17.737520 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-42e5c018-91bf-439a-888f-49b2f4ef0f94 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:17.740391 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-42e5c018-91bf-439a-888f-49b2f4ef0f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:17.751851 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-82193347-8f1d-466b-aacb-b47456b83409 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5324c3957e004c548e5fb243d6a23aaf', 'request_id': 'req-82193347-8f1d-466b-aacb-b47456b83409'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:17.761385 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-82193347-8f1d-466b-aacb-b47456b83409 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5324c3957e004c548e5fb243d6a23aaf', 'request_id': 'req-82193347-8f1d-466b-aacb-b47456b83409'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:17.785689 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-82193347-8f1d-466b-aacb-b47456b83409 admin admin] The token cache is being invalidate because project 5324c3957e004c548e5fb243d6a23aaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:10:17.785917 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-82193347-8f1d-466b-aacb-b47456b83409 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:17.786664 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-82193347-8f1d-466b-aacb-b47456b83409 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:17.789380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1019/2070] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:10:17 2022] DELETE /identity/v3/projects/5324c3957e004c548e5fb243d6a23aaf => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:17.794882 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef55f6e-a885-4faf-b677-6b4bb7297106 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:17.795313 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef55f6e-a885-4faf-b677-6b4bb7297106 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:17.795834 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef55f6e-a885-4faf-b677-6b4bb7297106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:17.837894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1052/2071] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:10:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:10:17.848274 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e5c018-91bf-439a-888f-49b2f4ef0f94 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:17.849742 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-0ef55f6e-a885-4faf-b677-6b4bb7297106 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:17.849911 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-42e5c018-91bf-439a-888f-49b2f4ef0f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:17.851067 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0ef55f6e-a885-4faf-b677-6b4bb7297106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:17.867326 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1020/2072] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:10:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:17.876527 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374764b9-40fa-4973-b91f-809f5be05e4f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:17.876840 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374764b9-40fa-4973-b91f-809f5be05e4f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:17.877518 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374764b9-40fa-4973-b91f-809f5be05e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:17.902718 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-374764b9-40fa-4973-b91f-809f5be05e4f None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:17.904231 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-374764b9-40fa-4973-b91f-809f5be05e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:17.912779 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1021/2073] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:10:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:17.917631 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374764b9-40fa-4973-b91f-809f5be05e4f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:17.928634 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374764b9-40fa-4973-b91f-809f5be05e4f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:17.937841 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6be83-6a05-4618-9c3d-8c7b30db3e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:17.938792 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6be83-6a05-4618-9c3d-8c7b30db3e09 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:17.939213 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6be83-6a05-4618-9c3d-8c7b30db3e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:17.939609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6be83-6a05-4618-9c3d-8c7b30db3e09 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:17.941567 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6be83-6a05-4618-9c3d-8c7b30db3e09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:10:17.941567 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6be83-6a05-4618-9c3d-8c7b30db3e09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:17.943473 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6be83-6a05-4618-9c3d-8c7b30db3e09 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:17.945713 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e5c018-91bf-439a-888f-49b2f4ef0f94 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:17.950453 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f1d24-2685-4652-a0a6-5381edbc70ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:17.951091 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f1d24-2685-4652-a0a6-5381edbc70ce service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:17.951425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f1d24-2685-4652-a0a6-5381edbc70ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:17.952088 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f1d24-2685-4652-a0a6-5381edbc70ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:17.958430 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1022/2074] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:10:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:10:17.963183 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6be83-6a05-4618-9c3d-8c7b30db3e09 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:17.963682 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f1d24-2685-4652-a0a6-5381edbc70ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:17.965326 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f1d24-2685-4652-a0a6-5381edbc70ce service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:17.974701 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6be83-6a05-4618-9c3d-8c7b30db3e09 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:17.979876 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1053/2075] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:10:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:10:17.980770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945c154-9873-467b-84b9-c36843e1fa65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:17.981608 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945c154-9873-467b-84b9-c36843e1fa65 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:17.982075 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945c154-9873-467b-84b9-c36843e1fa65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:17.982458 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945c154-9873-467b-84b9-c36843e1fa65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:17.983554 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c945c154-9873-467b-84b9-c36843e1fa65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:17.985453 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c945c154-9873-467b-84b9-c36843e1fa65 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:18.024142 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1023/2076] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:10:17 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:10:18.026365 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f1d24-2685-4652-a0a6-5381edbc70ce service neutron] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:18.040971 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f1d24-2685-4652-a0a6-5381edbc70ce service neutron] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:18.104449 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945c154-9873-467b-84b9-c36843e1fa65 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:18.106792 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c945c154-9873-467b-84b9-c36843e1fa65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:18.158073 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda87d11-6abc-40df-b416-5020610a828e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:18.159738 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda87d11-6abc-40df-b416-5020610a828e admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:18.160033 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda87d11-6abc-40df-b416-5020610a828e admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:18.160315 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda87d11-6abc-40df-b416-5020610a828e admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:18.161530 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda87d11-6abc-40df-b416-5020610a828e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:18.164881 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda87d11-6abc-40df-b416-5020610a828e admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:18.195390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:18.208853 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945c154-9873-467b-84b9-c36843e1fa65 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:18.219313 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9199a499-a1c9-4810-a940-4ce646f5f184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:18.220204 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199a499-a1c9-4810-a940-4ce646f5f184 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:18.220840 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199a499-a1c9-4810-a940-4ce646f5f184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:18.220984 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199a499-a1c9-4810-a940-4ce646f5f184 admin admin] PATH_INFO: `/v3/projects/7dfa8cf47ac2427596bc64c20e2224a9` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:18.222648 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9199a499-a1c9-4810-a940-4ce646f5f184 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dfa8cf47ac2427596bc64c20e2224a9)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:18.226021 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9199a499-a1c9-4810-a940-4ce646f5f184 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:18.229913 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1054/2077] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:10:18 2022] POST /identity/v3/users => generated 286 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:10:18.243790 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda87d11-6abc-40df-b416-5020610a828e admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:18.267282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda87d11-6abc-40df-b416-5020610a828e admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:18.284828 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecba1c8a-cf84-4269-ba38-b687499e228d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:18.284828 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba1c8a-cf84-4269-ba38-b687499e228d admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:18.284828 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba1c8a-cf84-4269-ba38-b687499e228d admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:18.284828 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba1c8a-cf84-4269-ba38-b687499e228d admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:18.285457 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecba1c8a-cf84-4269-ba38-b687499e228d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:18.285457 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecba1c8a-cf84-4269-ba38-b687499e228d admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:18.298837 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1055/2078] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:10:18.299764 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecba1c8a-cf84-4269-ba38-b687499e228d admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:18.312802 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecba1c8a-cf84-4269-ba38-b687499e228d admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:18.326598 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9199a499-a1c9-4810-a940-4ce646f5f184 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dfa8cf47ac2427596bc64c20e2224a9', 'request_id': 'req-9199a499-a1c9-4810-a940-4ce646f5f184'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:18.337301 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9199a499-a1c9-4810-a940-4ce646f5f184 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dfa8cf47ac2427596bc64c20e2224a9', 'request_id': 'req-9199a499-a1c9-4810-a940-4ce646f5f184'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:18.340023 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:18.351645 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9199a499-a1c9-4810-a940-4ce646f5f184 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dfa8cf47ac2427596bc64c20e2224a9', 'request_id': 'req-9199a499-a1c9-4810-a940-4ce646f5f184'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:18.370889 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9199a499-a1c9-4810-a940-4ce646f5f184 admin admin] The token cache is being invalidate because project 7dfa8cf47ac2427596bc64c20e2224a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:10:18.371683 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9199a499-a1c9-4810-a940-4ce646f5f184 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:18.372668 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9199a499-a1c9-4810-a940-4ce646f5f184 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:18.375381 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1024/2079] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:10:18 2022] DELETE /identity/v3/projects/7dfa8cf47ac2427596bc64c20e2224a9 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:10:18.381565 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dfd69f-b1eb-4384-af2b-3222829aed8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:18.382043 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524cd55b-7ec4-40cb-8b31-15d7d08becad None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:18.382184 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfd69f-b1eb-4384-af2b-3222829aed8f admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:18.382342 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524cd55b-7ec4-40cb-8b31-15d7d08becad None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:18.382468 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfd69f-b1eb-4384-af2b-3222829aed8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:18.382632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524cd55b-7ec4-40cb-8b31-15d7d08becad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:18.382738 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfd69f-b1eb-4384-af2b-3222829aed8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:18.383422 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dfd69f-b1eb-4384-af2b-3222829aed8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:18.385056 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dfd69f-b1eb-4384-af2b-3222829aed8f admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:18.396393 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1056/2080] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:10:18.400656 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dfd69f-b1eb-4384-af2b-3222829aed8f admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:18.402830 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-85dfd69f-b1eb-4384-af2b-3222829aed8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:18.407710 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-524cd55b-7ec4-40cb-8b31-15d7d08becad None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:18.409397 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-524cd55b-7ec4-40cb-8b31-15d7d08becad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:18.422358 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dfd69f-b1eb-4384-af2b-3222829aed8f admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:18.424628 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1025/2081] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:10:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:18.429249 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec675086-55d3-4d21-864b-996b8e9b8a08 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:18.429541 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec675086-55d3-4d21-864b-996b8e9b8a08 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:18.429889 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec675086-55d3-4d21-864b-996b8e9b8a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:18.436424 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c2a2f2-7502-4822-8c17-19e08aa10fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:18.437387 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2a2f2-7502-4822-8c17-19e08aa10fbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:18.437983 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2a2f2-7502-4822-8c17-19e08aa10fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:18.438433 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2a2f2-7502-4822-8c17-19e08aa10fbe admin admin] PATH_INFO: `/v3/projects/d0cc00c61d3e4e59b5790c99c6277d2d/users/7f415b2218694e828b0d160dfce5e114/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:18.459494 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ec675086-55d3-4d21-864b-996b8e9b8a08 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:18.459685 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c2a2f2-7502-4822-8c17-19e08aa10fbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0cc00c61d3e4e59b5790c99c6277d2d, user_id=7f415b2218694e828b0d160dfce5e114, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:18.461943 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ec675086-55d3-4d21-864b-996b8e9b8a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:18.469921 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c2a2f2-7502-4822-8c17-19e08aa10fbe admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:18.472111 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1026/2082] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:10:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:18.476698 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec675086-55d3-4d21-864b-996b8e9b8a08 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:18.491712 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1057/2083] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:10:18 2022] PUT /identity/v3/projects/d0cc00c61d3e4e59b5790c99c6277d2d/users/7f415b2218694e828b0d160dfce5e114/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:10:18.495644 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec675086-55d3-4d21-864b-996b8e9b8a08 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:18.502116 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955a0d4-ee19-4e60-8c94-fd080de1bc3b None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:18.502410 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955a0d4-ee19-4e60-8c94-fd080de1bc3b None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:18.502696 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955a0d4-ee19-4e60-8c94-fd080de1bc3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:18.534829 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-1955a0d4-ee19-4e60-8c94-fd080de1bc3b None None] MFA Rules not processed for user `7f415b2218694e828b0d160dfce5e114`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:18.537418 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-1955a0d4-ee19-4e60-8c94-fd080de1bc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:18.595693 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2dbf29-0234-4962-9fa6-22a2c750f935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:18.596248 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2dbf29-0234-4962-9fa6-22a2c750f935 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:18.596574 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2dbf29-0234-4962-9fa6-22a2c750f935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:18.597049 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2dbf29-0234-4962-9fa6-22a2c750f935 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:18.598159 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2dbf29-0234-4962-9fa6-22a2c750f935 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:10:18.599043 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2dbf29-0234-4962-9fa6-22a2c750f935 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:18.601238 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2dbf29-0234-4962-9fa6-22a2c750f935 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:18.616134 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1027/2084] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:10:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:10:18.621676 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2dbf29-0234-4962-9fa6-22a2c750f935 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:18.634244 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2dbf29-0234-4962-9fa6-22a2c750f935 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:18.636815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1058/2085] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:10:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:10:18.643351 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5cb770-439d-46b7-97c7-d91235f72cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:18.643993 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cb770-439d-46b7-97c7-d91235f72cc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:18.644279 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cb770-439d-46b7-97c7-d91235f72cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:18.644590 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cb770-439d-46b7-97c7-d91235f72cc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:18.645855 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5cb770-439d-46b7-97c7-d91235f72cc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:18.647762 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5cb770-439d-46b7-97c7-d91235f72cc7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:18.651200 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955a0d4-ee19-4e60-8c94-fd080de1bc3b None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:18.654098 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-1955a0d4-ee19-4e60-8c94-fd080de1bc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:18.682882 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1028/2086] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:10:18 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:10:18.686958 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5cb770-439d-46b7-97c7-d91235f72cc7 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:18.709853 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:18.729632 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955a0d4-ee19-4e60-8c94-fd080de1bc3b None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:18.763195 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5cb770-439d-46b7-97c7-d91235f72cc7 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:18.768561 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0cb46-b24e-4bd4-a073-758400897e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:18.768903 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0cb46-b24e-4bd4-a073-758400897e77 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:18.769154 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0cb46-b24e-4bd4-a073-758400897e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:18.769434 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0cb46-b24e-4bd4-a073-758400897e77 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:18.770119 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d0cb46-b24e-4bd4-a073-758400897e77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:18.772637 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d0cb46-b24e-4bd4-a073-758400897e77 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:18.794458 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a11827-24c1-4980-8ce9-7db65fe8f958 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:18.795097 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a11827-24c1-4980-8ce9-7db65fe8f958 service cinder] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:18.795434 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a11827-24c1-4980-8ce9-7db65fe8f958 service cinder] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:18.795718 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a11827-24c1-4980-8ce9-7db65fe8f958 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:18.798569 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1029/2087] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:10:18 2022] POST /identity/v3/users => generated 288 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:10:18.808734 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0cb46-b24e-4bd4-a073-758400897e77 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:18.820974 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0cb46-b24e-4bd4-a073-758400897e77 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:18.828700 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5d16dd-1803-4375-adf4-7d667cab2157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:18.829825 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d16dd-1803-4375-adf4-7d667cab2157 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:18.830278 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d16dd-1803-4375-adf4-7d667cab2157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:18.830702 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d16dd-1803-4375-adf4-7d667cab2157 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:18.831547 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5d16dd-1803-4375-adf4-7d667cab2157 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:18.834792 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5d16dd-1803-4375-adf4-7d667cab2157 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:18.851830 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1030/2088] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:10:18.852917 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5d16dd-1803-4375-adf4-7d667cab2157 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:18.864716 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5d16dd-1803-4375-adf4-7d667cab2157 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:18.870599 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5f104-d4b8-4725-b6c1-c79c4161c9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:18.871337 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5f104-d4b8-4725-b6c1-c79c4161c9a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:18.871516 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5f104-d4b8-4725-b6c1-c79c4161c9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:18.871784 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5f104-d4b8-4725-b6c1-c79c4161c9a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:18.872813 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd5f104-d4b8-4725-b6c1-c79c4161c9a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:18.875387 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd5f104-d4b8-4725-b6c1-c79c4161c9a7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:18.888532 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1031/2089] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:10:18.891813 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5f104-d4b8-4725-b6c1-c79c4161c9a7 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:18.892359 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a11827-24c1-4980-8ce9-7db65fe8f958 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:18.894272 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a11827-24c1-4980-8ce9-7db65fe8f958 service cinder] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:18.903390 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5f104-d4b8-4725-b6c1-c79c4161c9a7 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:18.908345 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc88e04-9763-4a08-ae81-a47a976f3e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:18.909069 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc88e04-9763-4a08-ae81-a47a976f3e2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:18.909458 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc88e04-9763-4a08-ae81-a47a976f3e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:18.909692 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc88e04-9763-4a08-ae81-a47a976f3e2e admin admin] PATH_INFO: `/v3/projects/4246e7b1e4c445bd9e8a2f58cb77959a/users/328cc8c14f87440998a642d797e9c3c1/roles/926b9f36c24847c582c84af904250abc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:18.916656 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1059/2090] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:10:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:10:18.919582 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:18.929626 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc88e04-9763-4a08-ae81-a47a976f3e2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4246e7b1e4c445bd9e8a2f58cb77959a, user_id=328cc8c14f87440998a642d797e9c3c1, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:18.932734 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc88e04-9763-4a08-ae81-a47a976f3e2e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:18.953930 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1032/2091] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:10:18 2022] PUT /identity/v3/projects/4246e7b1e4c445bd9e8a2f58cb77959a/users/328cc8c14f87440998a642d797e9c3c1/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:18.964811 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275a576-09b2-42a7-a128-30f2e4cbc86d None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:18.965104 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275a576-09b2-42a7-a128-30f2e4cbc86d None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:18.965394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275a576-09b2-42a7-a128-30f2e4cbc86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:19.013593 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-c275a576-09b2-42a7-a128-30f2e4cbc86d None None] MFA Rules not processed for user `328cc8c14f87440998a642d797e9c3c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:19.015243 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-c275a576-09b2-42a7-a128-30f2e4cbc86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:19.110125 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1060/2092] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:10:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:19.114669 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc88e04-9763-4a08-ae81-a47a976f3e2e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:19.219729 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc88e04-9763-4a08-ae81-a47a976f3e2e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:19.226794 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0171bdd8-d9c8-41a2-9492-ec1933ea76d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:19.227562 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171bdd8-d9c8-41a2-9492-ec1933ea76d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:19.227944 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171bdd8-d9c8-41a2-9492-ec1933ea76d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:19.228343 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171bdd8-d9c8-41a2-9492-ec1933ea76d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:19.229533 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0171bdd8-d9c8-41a2-9492-ec1933ea76d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:19.231778 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0171bdd8-d9c8-41a2-9492-ec1933ea76d2 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:19.279041 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c275a576-09b2-42a7-a128-30f2e4cbc86d None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:19.287795 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1033/2093] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:10:19 2022] POST /identity/v3/projects => generated 375 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:10:19.309273 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c275a576-09b2-42a7-a128-30f2e4cbc86d None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:19.387754 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:19.422847 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7c7fc-e45f-47ce-a704-482aa5a14ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:19.423502 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7c7fc-e45f-47ce-a704-482aa5a14ebd admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:19.423849 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7c7fc-e45f-47ce-a704-482aa5a14ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:19.424142 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7c7fc-e45f-47ce-a704-482aa5a14ebd admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:19.425445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c7c7fc-e45f-47ce-a704-482aa5a14ebd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:19.427347 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c7c7fc-e45f-47ce-a704-482aa5a14ebd admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:19.460306 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1061/2094] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:10:19 2022] POST /identity/v3/users => generated 288 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:10:19.462710 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0171bdd8-d9c8-41a2-9492-ec1933ea76d2 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:19.475422 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0171bdd8-d9c8-41a2-9492-ec1933ea76d2 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:19.482002 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c437203-9c9a-4446-b8e7-500c1dd68b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:19.482639 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c437203-9c9a-4446-b8e7-500c1dd68b94 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:19.482924 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c437203-9c9a-4446-b8e7-500c1dd68b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:19.483201 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c437203-9c9a-4446-b8e7-500c1dd68b94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:19.484172 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c437203-9c9a-4446-b8e7-500c1dd68b94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:19.486093 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c437203-9c9a-4446-b8e7-500c1dd68b94 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:19.511820 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1034/2095] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:19 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:10:19.514171 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7c7fc-e45f-47ce-a704-482aa5a14ebd admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:19.524389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7c7fc-e45f-47ce-a704-482aa5a14ebd admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:19.529581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9277f-90da-40e6-a593-45c77a5b34a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:19.530209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9277f-90da-40e6-a593-45c77a5b34a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:19.530501 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9277f-90da-40e6-a593-45c77a5b34a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:19.530805 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9277f-90da-40e6-a593-45c77a5b34a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:19.531697 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b9277f-90da-40e6-a593-45c77a5b34a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:19.533416 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b9277f-90da-40e6-a593-45c77a5b34a1 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:19.549441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1062/2096] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:10:19.550862 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c437203-9c9a-4446-b8e7-500c1dd68b94 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:19.562209 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c437203-9c9a-4446-b8e7-500c1dd68b94 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:19.568190 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488af131-6bc8-45e5-b076-c1fdf0189b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:19.568987 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488af131-6bc8-45e5-b076-c1fdf0189b4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:19.569271 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488af131-6bc8-45e5-b076-c1fdf0189b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:19.569553 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488af131-6bc8-45e5-b076-c1fdf0189b4a admin admin] PATH_INFO: `/v3/projects/91ac6b5e3235400181f06dc5efb8fa35/users/e60e34eaaba74d1da1877c9533a708a4/roles/926b9f36c24847c582c84af904250abc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:19.585963 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488af131-6bc8-45e5-b076-c1fdf0189b4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=91ac6b5e3235400181f06dc5efb8fa35, user_id=e60e34eaaba74d1da1877c9533a708a4, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:19.588819 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488af131-6bc8-45e5-b076-c1fdf0189b4a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:19.608556 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1035/2097] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:10:19 2022] PUT /identity/v3/projects/91ac6b5e3235400181f06dc5efb8fa35/users/e60e34eaaba74d1da1877c9533a708a4/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:10:19.615509 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9fa56-9100-47e2-935d-59801120579d None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:19.615713 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9fa56-9100-47e2-935d-59801120579d None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:19.616020 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9fa56-9100-47e2-935d-59801120579d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:19.653805 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-b0a9fa56-9100-47e2-935d-59801120579d None None] MFA Rules not processed for user `e60e34eaaba74d1da1877c9533a708a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:19.654967 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-b0a9fa56-9100-47e2-935d-59801120579d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:19.734388 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1063/2098] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:10:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:19.738669 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488af131-6bc8-45e5-b076-c1fdf0189b4a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:19.810980 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488af131-6bc8-45e5-b076-c1fdf0189b4a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:19.817317 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e4760-a394-4900-97ec-8dd64f2d52aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:19.817956 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e4760-a394-4900-97ec-8dd64f2d52aa admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:19.818244 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e4760-a394-4900-97ec-8dd64f2d52aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:19.820243 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e4760-a394-4900-97ec-8dd64f2d52aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:19.820243 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e4760-a394-4900-97ec-8dd64f2d52aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:19.823500 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e4760-a394-4900-97ec-8dd64f2d52aa admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:19.876175 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1036/2099] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:10:19 2022] POST /identity/v3/projects => generated 377 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:10:19.881109 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a9fa56-9100-47e2-935d-59801120579d None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:19.892661 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a9fa56-9100-47e2-935d-59801120579d None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:19.993637 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc8683-5d30-40d4-9d03-60d02b046928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:19.994302 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc8683-5d30-40d4-9d03-60d02b046928 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:19.994569 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc8683-5d30-40d4-9d03-60d02b046928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:19.994846 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc8683-5d30-40d4-9d03-60d02b046928 admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:19.995634 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fc8683-5d30-40d4-9d03-60d02b046928 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:19.997593 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fc8683-5d30-40d4-9d03-60d02b046928 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:20.047121 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1064/2100] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:10:19 2022] POST /identity/v3/users => generated 295 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:10:20.051019 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e4760-a394-4900-97ec-8dd64f2d52aa admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:20.064892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e4760-a394-4900-97ec-8dd64f2d52aa admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:20.075055 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ee0aa-24e3-45da-a7ad-bdb7db8803dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:20.075700 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ee0aa-24e3-45da-a7ad-bdb7db8803dd admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:20.075984 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ee0aa-24e3-45da-a7ad-bdb7db8803dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:20.076786 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ee0aa-24e3-45da-a7ad-bdb7db8803dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:20.080905 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ee0aa-24e3-45da-a7ad-bdb7db8803dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:20.081139 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ee0aa-24e3-45da-a7ad-bdb7db8803dd admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:20.094817 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1037/2101] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:10:20.101655 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc8683-5d30-40d4-9d03-60d02b046928 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:20.112848 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc8683-5d30-40d4-9d03-60d02b046928 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:20.117617 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b225ab8-8c6a-488e-8410-d9b10c0bb37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:20.118299 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b225ab8-8c6a-488e-8410-d9b10c0bb37b admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:20.118643 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b225ab8-8c6a-488e-8410-d9b10c0bb37b admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:20.118870 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b225ab8-8c6a-488e-8410-d9b10c0bb37b admin admin] PATH_INFO: `/v3/projects/5b18139bc5f74c4e9248c9428765e1b8/users/ed4090cfe5734c4a89be32f00c5636b8/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:20.133944 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b225ab8-8c6a-488e-8410-d9b10c0bb37b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b18139bc5f74c4e9248c9428765e1b8, user_id=ed4090cfe5734c4a89be32f00c5636b8, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:20.136827 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b225ab8-8c6a-488e-8410-d9b10c0bb37b admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:20.153471 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1065/2102] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:10:20 2022] PUT /identity/v3/projects/5b18139bc5f74c4e9248c9428765e1b8/users/ed4090cfe5734c4a89be32f00c5636b8/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:10:20.157392 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ee0aa-24e3-45da-a7ad-bdb7db8803dd admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:20.168267 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ee0aa-24e3-45da-a7ad-bdb7db8803dd admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:20.176502 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:20.238646 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c4d6f-45d0-41cd-99ac-19a491d5f76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:20.239436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c4d6f-45d0-41cd-99ac-19a491d5f76f admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:20.239724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c4d6f-45d0-41cd-99ac-19a491d5f76f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:20.240000 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c4d6f-45d0-41cd-99ac-19a491d5f76f admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:20.240772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c4d6f-45d0-41cd-99ac-19a491d5f76f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:20.242452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c4d6f-45d0-41cd-99ac-19a491d5f76f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:20.254292 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1038/2103] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:10:20.258394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b225ab8-8c6a-488e-8410-d9b10c0bb37b admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:20.269299 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b225ab8-8c6a-488e-8410-d9b10c0bb37b admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:20.274986 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986db164-a9a5-4623-9b39-0be2dd2a79e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:20.275827 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986db164-a9a5-4623-9b39-0be2dd2a79e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:20.276173 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986db164-a9a5-4623-9b39-0be2dd2a79e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:20.276783 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986db164-a9a5-4623-9b39-0be2dd2a79e9 admin admin] PATH_INFO: `/v3/projects/5b18139bc5f74c4e9248c9428765e1b8/users/ed4090cfe5734c4a89be32f00c5636b8/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:20.279028 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986db164-a9a5-4623-9b39-0be2dd2a79e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b18139bc5f74c4e9248c9428765e1b8, user_id=ed4090cfe5734c4a89be32f00c5636b8, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:20.281857 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986db164-a9a5-4623-9b39-0be2dd2a79e9 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:20.290459 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:20.296841 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1066/2104] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:10:20 2022] PUT /identity/v3/projects/5b18139bc5f74c4e9248c9428765e1b8/users/ed4090cfe5734c4a89be32f00c5636b8/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:20.308023 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb06f7-a90f-4c49-8f40-6b54fb9106d7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:20.308312 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb06f7-a90f-4c49-8f40-6b54fb9106d7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:20.308985 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb06f7-a90f-4c49-8f40-6b54fb9106d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:20.346683 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-18bb06f7-a90f-4c49-8f40-6b54fb9106d7 None None] MFA Rules not processed for user `ed4090cfe5734c4a89be32f00c5636b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:20.346683 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-18bb06f7-a90f-4c49-8f40-6b54fb9106d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:20.409522 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe61f18-0cc0-4882-a80d-6a80a1eca0e2 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:20.409930 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe61f18-0cc0-4882-a80d-6a80a1eca0e2 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:20.410275 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe61f18-0cc0-4882-a80d-6a80a1eca0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:20.467196 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-afe61f18-0cc0-4882-a80d-6a80a1eca0e2 None None] MFA Rules not processed for user `7f415b2218694e828b0d160dfce5e114`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:20.470108 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-afe61f18-0cc0-4882-a80d-6a80a1eca0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:20.492921 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1039/2105] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:10:20 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:10:20.506950 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb06f7-a90f-4c49-8f40-6b54fb9106d7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:20.509327 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-18bb06f7-a90f-4c49-8f40-6b54fb9106d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:20.598611 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1067/2106] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:10:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:20.611328 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb06f7-a90f-4c49-8f40-6b54fb9106d7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:20.615947 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85822583-aed4-44d4-9eb7-31a36a56c3b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:20.616608 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85822583-aed4-44d4-9eb7-31a36a56c3b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:20.617212 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85822583-aed4-44d4-9eb7-31a36a56c3b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:20.617358 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85822583-aed4-44d4-9eb7-31a36a56c3b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:20.688072 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:20.711920 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85822583-aed4-44d4-9eb7-31a36a56c3b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:20.715174 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85822583-aed4-44d4-9eb7-31a36a56c3b7 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:20.727536 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1040/2107] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:10:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:10:22.534006 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe61f18-0cc0-4882-a80d-6a80a1eca0e2 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:22.552209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe61f18-0cc0-4882-a80d-6a80a1eca0e2 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:22.559917 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc33ff-6168-4881-901e-7dd77df18658 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:22.560937 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc33ff-6168-4881-901e-7dd77df18658 service cinder] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:22.561078 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc33ff-6168-4881-901e-7dd77df18658 service cinder] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:22.561333 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc33ff-6168-4881-901e-7dd77df18658 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:22.705739 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbc33ff-6168-4881-901e-7dd77df18658 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:22.705739 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbc33ff-6168-4881-901e-7dd77df18658 service cinder] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:22.711348 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1068/2108] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:10:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:10:23.129501 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba83c71-dc78-4179-a3f5-d8c45c28c89e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:23.129979 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba83c71-dc78-4179-a3f5-d8c45c28c89e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:23.130079 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba83c71-dc78-4179-a3f5-d8c45c28c89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:23.174116 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-9ba83c71-dc78-4179-a3f5-d8c45c28c89e None None] MFA Rules not processed for user `b6ed2d0b015d4fe39d780306d04fba4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:23.175493 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9ba83c71-dc78-4179-a3f5-d8c45c28c89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:23.261953 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1041/2109] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:10:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:23.266520 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc33ff-6168-4881-901e-7dd77df18658 service cinder] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:23.267932 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-ccbc33ff-6168-4881-901e-7dd77df18658 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:23.360690 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc33ff-6168-4881-901e-7dd77df18658 service cinder] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:23.365549 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d6d648-0980-41c2-9764-d5ccb913ae39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:23.366186 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6d648-0980-41c2-9764-d5ccb913ae39 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:23.366610 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6d648-0980-41c2-9764-d5ccb913ae39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:23.366920 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6d648-0980-41c2-9764-d5ccb913ae39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:23.367618 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d6d648-0980-41c2-9764-d5ccb913ae39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:23.369646 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d6d648-0980-41c2-9764-d5ccb913ae39 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:23.407382 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1069/2110] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:10:23 2022] POST /identity/v3/projects => generated 383 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:10:23.413787 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba83c71-dc78-4179-a3f5-d8c45c28c89e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:23.423860 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d6d648-0980-41c2-9764-d5ccb913ae39 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:23.428771 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba83c71-dc78-4179-a3f5-d8c45c28c89e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:23.459124 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:23.486381 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d6d648-0980-41c2-9764-d5ccb913ae39 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:23.490240 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0c037-4a2a-4208-8dd3-d2fbc3e84427 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:23.490871 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0c037-4a2a-4208-8dd3-d2fbc3e84427 service cinder] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:23.491156 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0c037-4a2a-4208-8dd3-d2fbc3e84427 service cinder] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:23.491437 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0c037-4a2a-4208-8dd3-d2fbc3e84427 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:23.498809 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb80b7f-1c11-40a6-ac98-569f27c13b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:23.499586 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb80b7f-1c11-40a6-ac98-569f27c13b29 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:23.499976 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb80b7f-1c11-40a6-ac98-569f27c13b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:23.500357 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb80b7f-1c11-40a6-ac98-569f27c13b29 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:23.501502 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb80b7f-1c11-40a6-ac98-569f27c13b29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:23.503219 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb80b7f-1c11-40a6-ac98-569f27c13b29 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:23.532526 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1042/2111] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:10:23 2022] POST /identity/v3/users => generated 298 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:10:23.533927 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb80b7f-1c11-40a6-ac98-569f27c13b29 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:23.544543 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb80b7f-1c11-40a6-ac98-569f27c13b29 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:23.549923 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06829f5b-0ff5-4eb7-b72e-abb83f4fdb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:23.550742 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06829f5b-0ff5-4eb7-b72e-abb83f4fdb6b admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:23.551005 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06829f5b-0ff5-4eb7-b72e-abb83f4fdb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:23.551387 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06829f5b-0ff5-4eb7-b72e-abb83f4fdb6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:23.552168 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06829f5b-0ff5-4eb7-b72e-abb83f4fdb6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:23.553889 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06829f5b-0ff5-4eb7-b72e-abb83f4fdb6b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:23.565242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea0c037-4a2a-4208-8dd3-d2fbc3e84427 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:23.565837 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1043/2112] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:10:23.566559 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea0c037-4a2a-4208-8dd3-d2fbc3e84427 service cinder] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:23.569767 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06829f5b-0ff5-4eb7-b72e-abb83f4fdb6b admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:23.579561 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06829f5b-0ff5-4eb7-b72e-abb83f4fdb6b admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:23.582440 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1070/2113] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:10:23 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:10:23.593451 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8b594-d327-4ae9-834b-f26f9ae6ccf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:23.594237 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8b594-d327-4ae9-834b-f26f9ae6ccf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:23.594526 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8b594-d327-4ae9-834b-f26f9ae6ccf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:23.594813 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8b594-d327-4ae9-834b-f26f9ae6ccf3 admin admin] PATH_INFO: `/v3/projects/cac144718edb4e0aaf75e7443eae2bf0/users/e243686531424308bca5737102518edc/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:23.611601 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e8b594-d327-4ae9-834b-f26f9ae6ccf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cac144718edb4e0aaf75e7443eae2bf0, user_id=e243686531424308bca5737102518edc, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:23.614380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e8b594-d327-4ae9-834b-f26f9ae6ccf3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:23.629704 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1044/2114] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:10:23 2022] PUT /identity/v3/projects/cac144718edb4e0aaf75e7443eae2bf0/users/e243686531424308bca5737102518edc/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:23.634864 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0c037-4a2a-4208-8dd3-d2fbc3e84427 service cinder] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:23.657817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0c037-4a2a-4208-8dd3-d2fbc3e84427 service cinder] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:23.758894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9035858f-61d6-4f2d-a3cc-e88b6aec63c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:23.759928 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035858f-61d6-4f2d-a3cc-e88b6aec63c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:23.760437 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035858f-61d6-4f2d-a3cc-e88b6aec63c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:23.760979 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035858f-61d6-4f2d-a3cc-e88b6aec63c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:23.761898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9035858f-61d6-4f2d-a3cc-e88b6aec63c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:23.763748 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9035858f-61d6-4f2d-a3cc-e88b6aec63c9 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:23.775769 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1071/2115] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:23 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:10:23.779969 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8b594-d327-4ae9-834b-f26f9ae6ccf3 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:23.790796 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8b594-d327-4ae9-834b-f26f9ae6ccf3 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:23.795835 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7112980-0201-4a9b-9644-b5e9e80b80cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:23.796451 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7112980-0201-4a9b-9644-b5e9e80b80cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:23.797278 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7112980-0201-4a9b-9644-b5e9e80b80cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:23.797573 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7112980-0201-4a9b-9644-b5e9e80b80cf admin admin] PATH_INFO: `/v3/projects/cac144718edb4e0aaf75e7443eae2bf0/users/e243686531424308bca5737102518edc/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:23.799541 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7112980-0201-4a9b-9644-b5e9e80b80cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=cac144718edb4e0aaf75e7443eae2bf0, user_id=e243686531424308bca5737102518edc, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:23.802287 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7112980-0201-4a9b-9644-b5e9e80b80cf admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:23.814083 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1045/2116] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:10:23 2022] PUT /identity/v3/projects/cac144718edb4e0aaf75e7443eae2bf0/users/e243686531424308bca5737102518edc/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:28.134847 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9035858f-61d6-4f2d-a3cc-e88b6aec63c9 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:28.153695 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9035858f-61d6-4f2d-a3cc-e88b6aec63c9 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:28.349943 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6e8b6-e125-4f4a-8af8-7646345fdde7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:28.350611 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6e8b6-e125-4f4a-8af8-7646345fdde7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:28.351068 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6e8b6-e125-4f4a-8af8-7646345fdde7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:28.351517 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6e8b6-e125-4f4a-8af8-7646345fdde7 admin admin] PATH_INFO: `/v3/users/328cc8c14f87440998a642d797e9c3c1` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:28.354064 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb6e8b6-e125-4f4a-8af8-7646345fdde7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=328cc8c14f87440998a642d797e9c3c1)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:28.357072 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb6e8b6-e125-4f4a-8af8-7646345fdde7 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:28.467382 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:28.542793 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-4eb6e8b6-e125-4f4a-8af8-7646345fdde7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '328cc8c14f87440998a642d797e9c3c1'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:28.543747 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-4eb6e8b6-e125-4f4a-8af8-7646345fdde7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '328cc8c14f87440998a642d797e9c3c1'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:28.573016 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-4eb6e8b6-e125-4f4a-8af8-7646345fdde7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '328cc8c14f87440998a642d797e9c3c1'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:28.589348 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-4eb6e8b6-e125-4f4a-8af8-7646345fdde7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '328cc8c14f87440998a642d797e9c3c1'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:28.590003 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-4eb6e8b6-e125-4f4a-8af8-7646345fdde7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '328cc8c14f87440998a642d797e9c3c1'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:28.616307 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1072/2117] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:10:28 2022] DELETE /identity/v3/users/328cc8c14f87440998a642d797e9c3c1 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:28.620961 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6305c7c-2965-4274-85cc-f2834087ad4d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:28.621464 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6305c7c-2965-4274-85cc-f2834087ad4d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:28.621564 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6305c7c-2965-4274-85cc-f2834087ad4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:28.657672 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b6305c7c-2965-4274-85cc-f2834087ad4d None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:28.658951 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b6305c7c-2965-4274-85cc-f2834087ad4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:28.689328 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:28.790628 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1046/2118] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:10:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:28.823305 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6e8b6-e125-4f4a-8af8-7646345fdde7 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:28.824815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-4eb6e8b6-e125-4f4a-8af8-7646345fdde7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:28.974363 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6e8b6-e125-4f4a-8af8-7646345fdde7 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:28.993083 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470604bc-509a-4ae0-bf13-37ab90eaf521 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:28.994024 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470604bc-509a-4ae0-bf13-37ab90eaf521 service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:28.994414 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470604bc-509a-4ae0-bf13-37ab90eaf521 service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:28.994987 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470604bc-509a-4ae0-bf13-37ab90eaf521 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:29.005900 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470604bc-509a-4ae0-bf13-37ab90eaf521 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:29.007510 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470604bc-509a-4ae0-bf13-37ab90eaf521 service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:29.022076 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1073/2119] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:10:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:10:30.370404 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6305c7c-2965-4274-85cc-f2834087ad4d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:30.376115 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b6305c7c-2965-4274-85cc-f2834087ad4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:30.393333 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6305c7c-2965-4274-85cc-f2834087ad4d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:30.398689 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db786d86-b5d0-4cec-9bb4-56c03f3361ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:30.399404 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db786d86-b5d0-4cec-9bb4-56c03f3361ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:30.399751 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db786d86-b5d0-4cec-9bb4-56c03f3361ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:30.400214 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db786d86-b5d0-4cec-9bb4-56c03f3361ef admin admin] PATH_INFO: `/v3/projects/4246e7b1e4c445bd9e8a2f58cb77959a` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:30.401498 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db786d86-b5d0-4cec-9bb4-56c03f3361ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=4246e7b1e4c445bd9e8a2f58cb77959a)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:30.403544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db786d86-b5d0-4cec-9bb4-56c03f3361ef admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:30.484842 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-db786d86-b5d0-4cec-9bb4-56c03f3361ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4246e7b1e4c445bd9e8a2f58cb77959a', 'request_id': 'req-db786d86-b5d0-4cec-9bb4-56c03f3361ef'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:30.499674 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-db786d86-b5d0-4cec-9bb4-56c03f3361ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4246e7b1e4c445bd9e8a2f58cb77959a', 'request_id': 'req-db786d86-b5d0-4cec-9bb4-56c03f3361ef'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:30.511386 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-db786d86-b5d0-4cec-9bb4-56c03f3361ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4246e7b1e4c445bd9e8a2f58cb77959a', 'request_id': 'req-db786d86-b5d0-4cec-9bb4-56c03f3361ef'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:30.537280 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-db786d86-b5d0-4cec-9bb4-56c03f3361ef admin admin] The token cache is being invalidate because project 4246e7b1e4c445bd9e8a2f58cb77959a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:10:30.537584 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-db786d86-b5d0-4cec-9bb4-56c03f3361ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:30.542862 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-db786d86-b5d0-4cec-9bb4-56c03f3361ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:30.544606 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1047/2120] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:10:30 2022] DELETE /identity/v3/projects/4246e7b1e4c445bd9e8a2f58cb77959a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:30.548962 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470604bc-509a-4ae0-bf13-37ab90eaf521 service neutron] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:30.570914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470604bc-509a-4ae0-bf13-37ab90eaf521 service neutron] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:30.732239 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cf514f-c238-486d-bedf-e3c2a71f4f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:30.735413 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf514f-c238-486d-bedf-e3c2a71f4f89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:30.737085 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf514f-c238-486d-bedf-e3c2a71f4f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:30.737383 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf514f-c238-486d-bedf-e3c2a71f4f89 admin admin] PATH_INFO: `/v3/users/e60e34eaaba74d1da1877c9533a708a4` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:30.743624 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cf514f-c238-486d-bedf-e3c2a71f4f89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e60e34eaaba74d1da1877c9533a708a4)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:30.745302 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cf514f-c238-486d-bedf-e3c2a71f4f89 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:30.825827 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:30.836401 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-b6cf514f-c238-486d-bedf-e3c2a71f4f89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e60e34eaaba74d1da1877c9533a708a4'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:30.837613 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-b6cf514f-c238-486d-bedf-e3c2a71f4f89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e60e34eaaba74d1da1877c9533a708a4'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:30.856396 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-b6cf514f-c238-486d-bedf-e3c2a71f4f89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e60e34eaaba74d1da1877c9533a708a4'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:30.871681 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-b6cf514f-c238-486d-bedf-e3c2a71f4f89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e60e34eaaba74d1da1877c9533a708a4'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:30.882454 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-b6cf514f-c238-486d-bedf-e3c2a71f4f89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e60e34eaaba74d1da1877c9533a708a4'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:30.899789 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1074/2121] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:10:30 2022] DELETE /identity/v3/users/e60e34eaaba74d1da1877c9533a708a4 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:31.430736 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db786d86-b5d0-4cec-9bb4-56c03f3361ef admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:31.432345 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-db786d86-b5d0-4cec-9bb4-56c03f3361ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:31.446040 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db786d86-b5d0-4cec-9bb4-56c03f3361ef admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:31.544920 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f7d09f-6fab-4597-9a0e-8019b54a81c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:31.545932 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f7d09f-6fab-4597-9a0e-8019b54a81c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:31.546224 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f7d09f-6fab-4597-9a0e-8019b54a81c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:31.546507 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f7d09f-6fab-4597-9a0e-8019b54a81c3 admin admin] PATH_INFO: `/v3/projects/91ac6b5e3235400181f06dc5efb8fa35` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:31.549028 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f7d09f-6fab-4597-9a0e-8019b54a81c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91ac6b5e3235400181f06dc5efb8fa35)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:31.551199 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f7d09f-6fab-4597-9a0e-8019b54a81c3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:31.614306 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:31.621076 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-34f7d09f-6fab-4597-9a0e-8019b54a81c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91ac6b5e3235400181f06dc5efb8fa35', 'request_id': 'req-34f7d09f-6fab-4597-9a0e-8019b54a81c3'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:31.631105 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-34f7d09f-6fab-4597-9a0e-8019b54a81c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91ac6b5e3235400181f06dc5efb8fa35', 'request_id': 'req-34f7d09f-6fab-4597-9a0e-8019b54a81c3'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:31.638186 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-34f7d09f-6fab-4597-9a0e-8019b54a81c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91ac6b5e3235400181f06dc5efb8fa35', 'request_id': 'req-34f7d09f-6fab-4597-9a0e-8019b54a81c3'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:31.651890 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-34f7d09f-6fab-4597-9a0e-8019b54a81c3 admin admin] The token cache is being invalidate because project 91ac6b5e3235400181f06dc5efb8fa35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:10:31.652201 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-34f7d09f-6fab-4597-9a0e-8019b54a81c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:31.652899 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-34f7d09f-6fab-4597-9a0e-8019b54a81c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:31.654449 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1048/2122] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:10:31 2022] DELETE /identity/v3/projects/91ac6b5e3235400181f06dc5efb8fa35 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:31.658583 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cf514f-c238-486d-bedf-e3c2a71f4f89 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:31.661562 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-b6cf514f-c238-486d-bedf-e3c2a71f4f89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:31.678834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cf514f-c238-486d-bedf-e3c2a71f4f89 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:31.719105 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a102a-cfd6-4609-8714-d43c6ea79df3 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:31.719409 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a102a-cfd6-4609-8714-d43c6ea79df3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:31.719677 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a102a-cfd6-4609-8714-d43c6ea79df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:31.752708 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f37a102a-cfd6-4609-8714-d43c6ea79df3 None None] MFA Rules not processed for user `e243686531424308bca5737102518edc`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:31.754797 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f37a102a-cfd6-4609-8714-d43c6ea79df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:31.758315 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30018d0a-9bbd-4f92-aa6b-f048f7e08973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:31.759309 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30018d0a-9bbd-4f92-aa6b-f048f7e08973 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:31.759748 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30018d0a-9bbd-4f92-aa6b-f048f7e08973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:31.760057 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30018d0a-9bbd-4f92-aa6b-f048f7e08973 admin admin] PATH_INFO: `/v3/users/ed4090cfe5734c4a89be32f00c5636b8` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:31.761424 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30018d0a-9bbd-4f92-aa6b-f048f7e08973 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed4090cfe5734c4a89be32f00c5636b8)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:31.763516 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30018d0a-9bbd-4f92-aa6b-f048f7e08973 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:31.851839 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1049/2123] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:10:31 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:31.865654 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a102a-cfd6-4609-8714-d43c6ea79df3 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:31.867232 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f37a102a-cfd6-4609-8714-d43c6ea79df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:31.872985 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-30018d0a-9bbd-4f92-aa6b-f048f7e08973 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed4090cfe5734c4a89be32f00c5636b8'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:31.873698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-30018d0a-9bbd-4f92-aa6b-f048f7e08973 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed4090cfe5734c4a89be32f00c5636b8'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:31.890362 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-30018d0a-9bbd-4f92-aa6b-f048f7e08973 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed4090cfe5734c4a89be32f00c5636b8'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:31.903953 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-30018d0a-9bbd-4f92-aa6b-f048f7e08973 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed4090cfe5734c4a89be32f00c5636b8'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:31.905388 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-30018d0a-9bbd-4f92-aa6b-f048f7e08973 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed4090cfe5734c4a89be32f00c5636b8'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:31.921639 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1075/2124] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:10:31 2022] DELETE /identity/v3/users/ed4090cfe5734c4a89be32f00c5636b8 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:31.937122 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a102a-cfd6-4609-8714-d43c6ea79df3 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:32.027299 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ded650-00d7-4267-ad16-0dffa5654aa5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39a66f7362964389be039937a786bb0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:32.028084 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ded650-00d7-4267-ad16-0dffa5654aa5 service glance] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:32.028453 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ded650-00d7-4267-ad16-0dffa5654aa5 service glance] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:32.028878 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ded650-00d7-4267-ad16-0dffa5654aa5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:32.032056 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a3ded650-00d7-4267-ad16-0dffa5654aa5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:32.070098 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:32.116779 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ded650-00d7-4267-ad16-0dffa5654aa5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:32.118236 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ded650-00d7-4267-ad16-0dffa5654aa5 service glance] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:32.141502 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1050/2125] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:10:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:10:32.293837 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30018d0a-9bbd-4f92-aa6b-f048f7e08973 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:32.296376 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-30018d0a-9bbd-4f92-aa6b-f048f7e08973 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:32.322250 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb2098-a9d7-4ca3-a016-d13e69deb93b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:32.322648 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb2098-a9d7-4ca3-a016-d13e69deb93b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:32.323001 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb2098-a9d7-4ca3-a016-d13e69deb93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:32.344744 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-61eb2098-a9d7-4ca3-a016-d13e69deb93b None None] MFA Rules not processed for user `5007a2f8e6244b0788f0a554cc572fe5`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:32.346132 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-61eb2098-a9d7-4ca3-a016-d13e69deb93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:32.348674 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:32.406620 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30018d0a-9bbd-4f92-aa6b-f048f7e08973 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:32.415025 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ca05d-f06a-4ed9-8109-399482836bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:32.416628 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ca05d-f06a-4ed9-8109-399482836bc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:32.417014 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ca05d-f06a-4ed9-8109-399482836bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:32.417304 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ca05d-f06a-4ed9-8109-399482836bc6 admin admin] PATH_INFO: `/v3/projects/5b18139bc5f74c4e9248c9428765e1b8` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:32.418554 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ca05d-f06a-4ed9-8109-399482836bc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b18139bc5f74c4e9248c9428765e1b8)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:32.421841 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ca05d-f06a-4ed9-8109-399482836bc6 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:32.443398 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1051/2126] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 14:10:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:10:32.508217 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb2098-a9d7-4ca3-a016-d13e69deb93b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:32.510214 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-61eb2098-a9d7-4ca3-a016-d13e69deb93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:32.518905 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d69ca05d-f06a-4ed9-8109-399482836bc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b18139bc5f74c4e9248c9428765e1b8', 'request_id': 'req-d69ca05d-f06a-4ed9-8109-399482836bc6'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:32.529188 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d69ca05d-f06a-4ed9-8109-399482836bc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b18139bc5f74c4e9248c9428765e1b8', 'request_id': 'req-d69ca05d-f06a-4ed9-8109-399482836bc6'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:32.536989 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d69ca05d-f06a-4ed9-8109-399482836bc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b18139bc5f74c4e9248c9428765e1b8', 'request_id': 'req-d69ca05d-f06a-4ed9-8109-399482836bc6'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:32.555103 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d69ca05d-f06a-4ed9-8109-399482836bc6 admin admin] The token cache is being invalidate because project 5b18139bc5f74c4e9248c9428765e1b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:10:32.555500 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d69ca05d-f06a-4ed9-8109-399482836bc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:32.556109 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d69ca05d-f06a-4ed9-8109-399482836bc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:32.557929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1076/2127] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:10:32 2022] DELETE /identity/v3/projects/5b18139bc5f74c4e9248c9428765e1b8 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:32.564347 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648a8e2-9989-4bb5-839b-4b66c2e7ebb3 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:32.564653 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648a8e2-9989-4bb5-839b-4b66c2e7ebb3 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:32.564941 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648a8e2-9989-4bb5-839b-4b66c2e7ebb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:32.588786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-c648a8e2-9989-4bb5-839b-4b66c2e7ebb3 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:32.590782 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-c648a8e2-9989-4bb5-839b-4b66c2e7ebb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:32.636978 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb2098-a9d7-4ca3-a016-d13e69deb93b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:32.659259 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1077/2128] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:10:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:32.667501 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474dd580-1ae5-4cf0-bde5-0611e42e5e1d None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:32.667797 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474dd580-1ae5-4cf0-bde5-0611e42e5e1d None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:32.668208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474dd580-1ae5-4cf0-bde5-0611e42e5e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:32.693513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-474dd580-1ae5-4cf0-bde5-0611e42e5e1d None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:32.695507 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-474dd580-1ae5-4cf0-bde5-0611e42e5e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:32.707143 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1078/2129] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:10:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:10:32.708425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474dd580-1ae5-4cf0-bde5-0611e42e5e1d None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:32.717748 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474dd580-1ae5-4cf0-bde5-0611e42e5e1d None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:32.718605 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd230f-797b-4354-8668-ba8c134761a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2a6d7bd10435ba2c8e552d808b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:32.719243 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd230f-797b-4354-8668-ba8c134761a7 service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:32.719580 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd230f-797b-4354-8668-ba8c134761a7 service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:32.719749 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd230f-797b-4354-8668-ba8c134761a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:32.721901 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b4fd230f-797b-4354-8668-ba8c134761a7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:32.722470 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd7b9f8-e651-4896-91a1-9aa7d6486ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:32.723082 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7b9f8-e651-4896-91a1-9aa7d6486ab7 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:32.723362 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7b9f8-e651-4896-91a1-9aa7d6486ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:32.723633 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7b9f8-e651-4896-91a1-9aa7d6486ab7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:32.724336 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd7b9f8-e651-4896-91a1-9aa7d6486ab7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:10:32.724682 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd7b9f8-e651-4896-91a1-9aa7d6486ab7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:32.726312 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd7b9f8-e651-4896-91a1-9aa7d6486ab7 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:32.736496 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:32.744367 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1079/2130] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:10:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:10:32.746024 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd7b9f8-e651-4896-91a1-9aa7d6486ab7 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:32.759188 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd7b9f8-e651-4896-91a1-9aa7d6486ab7 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:32.764656 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fcacbc-1613-4323-bc47-f8cd09508929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:32.765742 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcacbc-1613-4323-bc47-f8cd09508929 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:32.765742 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcacbc-1613-4323-bc47-f8cd09508929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:32.765929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcacbc-1613-4323-bc47-f8cd09508929 admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:32.766511 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fcacbc-1613-4323-bc47-f8cd09508929 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:32.768409 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fcacbc-1613-4323-bc47-f8cd09508929 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:32.791169 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fd230f-797b-4354-8668-ba8c134761a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:32.792914 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fd230f-797b-4354-8668-ba8c134761a7 service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:32.801297 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1080/2131] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:10:32 2022] POST /identity/v3/projects => generated 369 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:10:32.805722 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fcacbc-1613-4323-bc47-f8cd09508929 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:32.808194 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1052/2132] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 14:10:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 01 14:10:32.844908 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:32.905142 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fcacbc-1613-4323-bc47-f8cd09508929 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:32.912496 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfe9b10-dddf-45c8-9cca-57aa70d27c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:32.913143 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe9b10-dddf-45c8-9cca-57aa70d27c6b admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:32.913428 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe9b10-dddf-45c8-9cca-57aa70d27c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:32.913708 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe9b10-dddf-45c8-9cca-57aa70d27c6b admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:32.914385 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfe9b10-dddf-45c8-9cca-57aa70d27c6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:32.916572 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfe9b10-dddf-45c8-9cca-57aa70d27c6b admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:32.969419 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1081/2133] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:10:32 2022] POST /identity/v3/users => generated 285 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:10:32.972687 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd230f-797b-4354-8668-ba8c134761a7 service swift] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:32.989106 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd230f-797b-4354-8668-ba8c134761a7 service swift] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:32.994194 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ea353-b711-4cfa-afb4-765b29819efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:32.995073 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ea353-b711-4cfa-afb4-765b29819efb admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:32.995616 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ea353-b711-4cfa-afb4-765b29819efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:32.996132 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ea353-b711-4cfa-afb4-765b29819efb admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:32.996951 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244ea353-b711-4cfa-afb4-765b29819efb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:32.999369 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244ea353-b711-4cfa-afb4-765b29819efb admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:33.010759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1053/2134] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:10:33.014576 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfe9b10-dddf-45c8-9cca-57aa70d27c6b admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:33.034092 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfe9b10-dddf-45c8-9cca-57aa70d27c6b admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:33.043469 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd6bcb-9551-44ca-b350-60b12fc2d3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:33.044103 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd6bcb-9551-44ca-b350-60b12fc2d3b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:33.044389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd6bcb-9551-44ca-b350-60b12fc2d3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:33.044695 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd6bcb-9551-44ca-b350-60b12fc2d3b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:33.045406 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd6bcb-9551-44ca-b350-60b12fc2d3b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:33.047013 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd6bcb-9551-44ca-b350-60b12fc2d3b1 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:33.071084 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1082/2135] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:33 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:10:33.071295 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ea353-b711-4cfa-afb4-765b29819efb admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:33.095017 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ea353-b711-4cfa-afb4-765b29819efb admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:33.117759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f101c78d-4967-481e-996c-0a2fec0e9f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:33.118462 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101c78d-4967-481e-996c-0a2fec0e9f36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:33.118754 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101c78d-4967-481e-996c-0a2fec0e9f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:33.119043 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101c78d-4967-481e-996c-0a2fec0e9f36 admin admin] PATH_INFO: `/v3/projects/7808386b690a4255b84d794c761dd024/users/4dcdf6d6c1fe4137bb0995ae30b1d4f9/roles/926b9f36c24847c582c84af904250abc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:33.150770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f101c78d-4967-481e-996c-0a2fec0e9f36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7808386b690a4255b84d794c761dd024, user_id=4dcdf6d6c1fe4137bb0995ae30b1d4f9, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:33.154175 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f101c78d-4967-481e-996c-0a2fec0e9f36 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:33.178939 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1054/2136] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:10:33 2022] PUT /identity/v3/projects/7808386b690a4255b84d794c761dd024/users/4dcdf6d6c1fe4137bb0995ae30b1d4f9/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:33.194678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcdac0-0ff5-4b1d-8f16-8b7c01ebd41c None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:33.195209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcdac0-0ff5-4b1d-8f16-8b7c01ebd41c None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:33.195655 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcdac0-0ff5-4b1d-8f16-8b7c01ebd41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:33.244964 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-13bcdac0-0ff5-4b1d-8f16-8b7c01ebd41c None None] MFA Rules not processed for user `4dcdf6d6c1fe4137bb0995ae30b1d4f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:33.250766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-13bcdac0-0ff5-4b1d-8f16-8b7c01ebd41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:33.380823 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1083/2137] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:10:33 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:33.395238 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f101c78d-4967-481e-996c-0a2fec0e9f36 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:33.397271 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f101c78d-4967-481e-996c-0a2fec0e9f36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:33.490184 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f101c78d-4967-481e-996c-0a2fec0e9f36 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:33.493968 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dfc9bb-92c3-4f5e-aa35-536f9bf8db7d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39a66f7362964389be039937a786bb0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:33.494597 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dfc9bb-92c3-4f5e-aa35-536f9bf8db7d service glance] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:33.494876 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dfc9bb-92c3-4f5e-aa35-536f9bf8db7d service glance] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:33.495147 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dfc9bb-92c3-4f5e-aa35-536f9bf8db7d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:33.517396 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dfc9bb-92c3-4f5e-aa35-536f9bf8db7d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:33.519662 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dfc9bb-92c3-4f5e-aa35-536f9bf8db7d service glance] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:33.537448 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1055/2138] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:10:33 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:10:41.417676 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b875b0-e506-42e6-a54f-7ea12cdca839 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:41.417676 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b875b0-e506-42e6-a54f-7ea12cdca839 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:41.418364 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b875b0-e506-42e6-a54f-7ea12cdca839 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:41.419092 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1084/2139] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 14:10:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:10:41.730183 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff9f1f-9056-4f52-a99d-f26ee7f7d563 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:41.731400 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff9f1f-9056-4f52-a99d-f26ee7f7d563 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:41.734224 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff9f1f-9056-4f52-a99d-f26ee7f7d563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:41.800019 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b875b0-e506-42e6-a54f-7ea12cdca839 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:41.802730 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-21b875b0-e506-42e6-a54f-7ea12cdca839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:41.837179 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4bff9f1f-9056-4f52-a99d-f26ee7f7d563 None None] MFA Rules not processed for user `e294fe796a1a4ac89041a532856ad627`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:41.838903 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4bff9f1f-9056-4f52-a99d-f26ee7f7d563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:41.962082 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b875b0-e506-42e6-a54f-7ea12cdca839 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:41.968986 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35125469-9417-4554-8aa5-42c39cdb1e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:41.969979 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35125469-9417-4554-8aa5-42c39cdb1e06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:41.970437 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35125469-9417-4554-8aa5-42c39cdb1e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:41.970874 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35125469-9417-4554-8aa5-42c39cdb1e06 admin admin] PATH_INFO: `/v3/users/822ea58eeb4e4d29b132964c0e3355db` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:41.972277 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35125469-9417-4554-8aa5-42c39cdb1e06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=822ea58eeb4e4d29b132964c0e3355db)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:41.974424 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35125469-9417-4554-8aa5-42c39cdb1e06 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:41.997171 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1056/2140] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 14:10:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:10:42.010537 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff9f1f-9056-4f52-a99d-f26ee7f7d563 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:42.011940 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4bff9f1f-9056-4f52-a99d-f26ee7f7d563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:42.024893 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:42.029152 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff9f1f-9056-4f52-a99d-f26ee7f7d563 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:42.038714 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe87bc91-b06b-43e8-91b9-8399e6427ada service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e294fe796a1a4ac89041a532856ad627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:42.039555 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87bc91-b06b-43e8-91b9-8399e6427ada service placement] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:42.040024 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87bc91-b06b-43e8-91b9-8399e6427ada service placement] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:42.040496 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87bc91-b06b-43e8-91b9-8399e6427ada service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:42.042048 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-35125469-9417-4554-8aa5-42c39cdb1e06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '822ea58eeb4e4d29b132964c0e3355db'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:42.046917 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-35125469-9417-4554-8aa5-42c39cdb1e06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '822ea58eeb4e4d29b132964c0e3355db'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:42.056109 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe87bc91-b06b-43e8-91b9-8399e6427ada service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:42.061009 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe87bc91-b06b-43e8-91b9-8399e6427ada service placement] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:42.074523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-35125469-9417-4554-8aa5-42c39cdb1e06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '822ea58eeb4e4d29b132964c0e3355db'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:42.081617 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1057/2141] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:10:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:10:42.094786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-35125469-9417-4554-8aa5-42c39cdb1e06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '822ea58eeb4e4d29b132964c0e3355db'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:42.097016 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-35125469-9417-4554-8aa5-42c39cdb1e06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '822ea58eeb4e4d29b132964c0e3355db'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:42.118970 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1085/2142] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:10:41 2022] DELETE /identity/v3/users/822ea58eeb4e4d29b132964c0e3355db => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:42.350383 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe87bc91-b06b-43e8-91b9-8399e6427ada service placement] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:42.365103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe87bc91-b06b-43e8-91b9-8399e6427ada service placement] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:42.368976 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:42.455452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15087cb4-ce8c-4ddc-ab97-81f54e04c8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:42.456410 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15087cb4-ce8c-4ddc-ab97-81f54e04c8a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:42.456896 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15087cb4-ce8c-4ddc-ab97-81f54e04c8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:42.457175 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15087cb4-ce8c-4ddc-ab97-81f54e04c8a6 admin admin] PATH_INFO: `/v3/projects/c68666a08e3142de947cfa44ec5816a9` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:42.458381 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15087cb4-ce8c-4ddc-ab97-81f54e04c8a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c68666a08e3142de947cfa44ec5816a9)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:42.460625 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15087cb4-ce8c-4ddc-ab97-81f54e04c8a6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:42.515531 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-15087cb4-ce8c-4ddc-ab97-81f54e04c8a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c68666a08e3142de947cfa44ec5816a9', 'request_id': 'req-15087cb4-ce8c-4ddc-ab97-81f54e04c8a6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:42.523939 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-15087cb4-ce8c-4ddc-ab97-81f54e04c8a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c68666a08e3142de947cfa44ec5816a9', 'request_id': 'req-15087cb4-ce8c-4ddc-ab97-81f54e04c8a6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:42.531412 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-15087cb4-ce8c-4ddc-ab97-81f54e04c8a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c68666a08e3142de947cfa44ec5816a9', 'request_id': 'req-15087cb4-ce8c-4ddc-ab97-81f54e04c8a6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:42.550467 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-15087cb4-ce8c-4ddc-ab97-81f54e04c8a6 admin admin] The token cache is being invalidate because project c68666a08e3142de947cfa44ec5816a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:10:42.550681 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-15087cb4-ce8c-4ddc-ab97-81f54e04c8a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:42.551312 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-15087cb4-ce8c-4ddc-ab97-81f54e04c8a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:42.553161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1058/2143] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:10:42 2022] DELETE /identity/v3/projects/c68666a08e3142de947cfa44ec5816a9 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:42.558065 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf47a22-68fd-4f6c-a235-bcbfd62f4368 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:42.558540 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf47a22-68fd-4f6c-a235-bcbfd62f4368 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:42.558637 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf47a22-68fd-4f6c-a235-bcbfd62f4368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:42.583283 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-fdf47a22-68fd-4f6c-a235-bcbfd62f4368 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:42.586894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-fdf47a22-68fd-4f6c-a235-bcbfd62f4368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:42.668722 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1086/2144] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:10:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:42.677384 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15ded8-dbaa-427e-b46e-c55768f90a63 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:42.677677 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15ded8-dbaa-427e-b46e-c55768f90a63 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:42.677958 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15ded8-dbaa-427e-b46e-c55768f90a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:42.700922 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-0b15ded8-dbaa-427e-b46e-c55768f90a63 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:42.701151 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0b15ded8-dbaa-427e-b46e-c55768f90a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:42.709405 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1059/2145] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:10:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:42.713733 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf47a22-68fd-4f6c-a235-bcbfd62f4368 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:42.725384 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf47a22-68fd-4f6c-a235-bcbfd62f4368 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:42.730511 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466d5d6b-e837-4f7b-a3a5-11913a4a333f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:42.731135 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d5d6b-e837-4f7b-a3a5-11913a4a333f admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:42.731421 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d5d6b-e837-4f7b-a3a5-11913a4a333f admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:42.731699 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d5d6b-e837-4f7b-a3a5-11913a4a333f admin admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:42.732406 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466d5d6b-e837-4f7b-a3a5-11913a4a333f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:10:42.732764 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466d5d6b-e837-4f7b-a3a5-11913a4a333f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:42.734369 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466d5d6b-e837-4f7b-a3a5-11913a4a333f admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:42.748591 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1087/2146] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:10:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:10:42.752830 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b15ded8-dbaa-427e-b46e-c55768f90a63 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:42.764770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b15ded8-dbaa-427e-b46e-c55768f90a63 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:42.769559 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66683601-091d-4772-be08-d43aee23c190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:42.770186 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66683601-091d-4772-be08-d43aee23c190 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:42.770472 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66683601-091d-4772-be08-d43aee23c190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:42.770750 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66683601-091d-4772-be08-d43aee23c190 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:42.771442 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66683601-091d-4772-be08-d43aee23c190 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:42.773391 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66683601-091d-4772-be08-d43aee23c190 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:42.811921 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1060/2147] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:10:42 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:10:42.816305 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466d5d6b-e837-4f7b-a3a5-11913a4a333f admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:42.837411 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466d5d6b-e837-4f7b-a3a5-11913a4a333f admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:42.930785 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd56536-4c33-4a72-9c81-3cfca40424c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:42.931422 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd56536-4c33-4a72-9c81-3cfca40424c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:42.931706 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd56536-4c33-4a72-9c81-3cfca40424c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:42.935095 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd56536-4c33-4a72-9c81-3cfca40424c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:42.936092 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd56536-4c33-4a72-9c81-3cfca40424c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:42.938120 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd56536-4c33-4a72-9c81-3cfca40424c0 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:42.974628 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1088/2148] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:10:42 2022] POST /identity/v3/users => generated 289 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:10:42.978884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66683601-091d-4772-be08-d43aee23c190 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:43.002778 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66683601-091d-4772-be08-d43aee23c190 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:43.010091 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997198ad-aa32-4bc3-96bc-26949225a32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:43.010942 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997198ad-aa32-4bc3-96bc-26949225a32a admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:43.011430 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997198ad-aa32-4bc3-96bc-26949225a32a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:43.024528 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997198ad-aa32-4bc3-96bc-26949225a32a admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:43.024528 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997198ad-aa32-4bc3-96bc-26949225a32a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:43.024528 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997198ad-aa32-4bc3-96bc-26949225a32a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:43.040232 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1061/2149] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:42 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:10:43.044002 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd56536-4c33-4a72-9c81-3cfca40424c0 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:43.070248 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd56536-4c33-4a72-9c81-3cfca40424c0 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:43.082740 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17540cdf-aded-4bf6-a83b-50e8692bf7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:43.083594 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17540cdf-aded-4bf6-a83b-50e8692bf7dd admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:43.083992 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17540cdf-aded-4bf6-a83b-50e8692bf7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:43.084373 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17540cdf-aded-4bf6-a83b-50e8692bf7dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:43.085216 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17540cdf-aded-4bf6-a83b-50e8692bf7dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:43.087598 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17540cdf-aded-4bf6-a83b-50e8692bf7dd admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:43.123697 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1089/2150] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:43 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:10:43.128339 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997198ad-aa32-4bc3-96bc-26949225a32a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:43.151655 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997198ad-aa32-4bc3-96bc-26949225a32a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:43.166392 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470989e-a234-40a6-a135-5c6df204e87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:43.167157 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470989e-a234-40a6-a135-5c6df204e87e admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:43.167445 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470989e-a234-40a6-a135-5c6df204e87e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:43.167768 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470989e-a234-40a6-a135-5c6df204e87e admin admin] PATH_INFO: `/v3/projects/c3d2b7b0783c4cc7bd0ce59a13f70efa/users/ebe4b3455a3d4139a1267da97bd83662/roles/926b9f36c24847c582c84af904250abc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:43.200499 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7470989e-a234-40a6-a135-5c6df204e87e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3d2b7b0783c4cc7bd0ce59a13f70efa, user_id=ebe4b3455a3d4139a1267da97bd83662, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:43.203224 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7470989e-a234-40a6-a135-5c6df204e87e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:43.234713 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1062/2151] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:10:43 2022] PUT /identity/v3/projects/c3d2b7b0783c4cc7bd0ce59a13f70efa/users/ebe4b3455a3d4139a1267da97bd83662/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:43.248932 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f52001-2304-4505-af24-28d2d044441f None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:43.249170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f52001-2304-4505-af24-28d2d044441f None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:43.249413 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f52001-2304-4505-af24-28d2d044441f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:43.270142 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-05f52001-2304-4505-af24-28d2d044441f None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:43.271426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-05f52001-2304-4505-af24-28d2d044441f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:43.310184 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:43.375462 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1090/2152] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:10:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:10:43.391759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470989e-a234-40a6-a135-5c6df204e87e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:43.395417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7470989e-a234-40a6-a135-5c6df204e87e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:43.452985 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:43.489792 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470989e-a234-40a6-a135-5c6df204e87e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:43.494324 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33da85c-815c-4b22-86a1-235350da3092 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:43.495126 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33da85c-815c-4b22-86a1-235350da3092 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:43.495418 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33da85c-815c-4b22-86a1-235350da3092 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:43.495834 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33da85c-815c-4b22-86a1-235350da3092 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:43.509224 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33da85c-815c-4b22-86a1-235350da3092 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:43.510707 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33da85c-815c-4b22-86a1-235350da3092 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:43.524159 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1063/2153] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:10:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:10:44.719393 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e1225-9f02-4e5d-9663-0428eeedd8e6 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:44.720006 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e1225-9f02-4e5d-9663-0428eeedd8e6 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:44.720006 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e1225-9f02-4e5d-9663-0428eeedd8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:44.749222 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-e24e1225-9f02-4e5d-9663-0428eeedd8e6 None None] MFA Rules not processed for user `b6ed2d0b015d4fe39d780306d04fba4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:44.750864 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-e24e1225-9f02-4e5d-9663-0428eeedd8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:44.848521 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1091/2154] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:10:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:48.967203 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839c8d1-0af9-4116-b5a0-9612d3b7a962 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:48.967819 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839c8d1-0af9-4116-b5a0-9612d3b7a962 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:48.967819 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839c8d1-0af9-4116-b5a0-9612d3b7a962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:49.008732 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e839c8d1-0af9-4116-b5a0-9612d3b7a962 None None] MFA Rules not processed for user `ebe4b3455a3d4139a1267da97bd83662`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:49.010306 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e839c8d1-0af9-4116-b5a0-9612d3b7a962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:49.105030 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1064/2155] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:10:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:10:49.118759 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e1225-9f02-4e5d-9663-0428eeedd8e6 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:49.120255 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-e24e1225-9f02-4e5d-9663-0428eeedd8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:49.202293 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e1225-9f02-4e5d-9663-0428eeedd8e6 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:49.208746 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa21734-8e60-4715-9d64-6c94cfec5900 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:49.209605 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa21734-8e60-4715-9d64-6c94cfec5900 service nova] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:49.209967 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa21734-8e60-4715-9d64-6c94cfec5900 service nova] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:49.210320 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa21734-8e60-4715-9d64-6c94cfec5900 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:49.219884 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa21734-8e60-4715-9d64-6c94cfec5900 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:49.222240 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa21734-8e60-4715-9d64-6c94cfec5900 service nova] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:49.233198 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1092/2156] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:10:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:10:50.255347 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0161306-beb3-4647-b02f-8590948bed88 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:50.256356 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0161306-beb3-4647-b02f-8590948bed88 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:50.256664 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0161306-beb3-4647-b02f-8590948bed88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:50.287295 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d0161306-beb3-4647-b02f-8590948bed88 None None] MFA Rules not processed for user `5007a2f8e6244b0788f0a554cc572fe5`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:50.290474 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d0161306-beb3-4647-b02f-8590948bed88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:50.410639 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1065/2157] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 14:10:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:10:50.429171 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa21734-8e60-4715-9d64-6c94cfec5900 service nova] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:50.431776 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-9aa21734-8e60-4715-9d64-6c94cfec5900 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:50.549834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:50.621475 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa21734-8e60-4715-9d64-6c94cfec5900 service nova] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:50.639493 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602b8ae-8e49-4326-9005-d8a0b3f8557d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2a6d7bd10435ba2c8e552d808b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:50.640390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602b8ae-8e49-4326-9005-d8a0b3f8557d service swift] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:50.640851 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602b8ae-8e49-4326-9005-d8a0b3f8557d service swift] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:50.641297 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602b8ae-8e49-4326-9005-d8a0b3f8557d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:50.659459 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9602b8ae-8e49-4326-9005-d8a0b3f8557d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:50.663816 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9602b8ae-8e49-4326-9005-d8a0b3f8557d service swift] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:50.686348 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1093/2158] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 14:10:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:10:51.319021 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ac478-eb15-4670-8b61-9d82632bf4aa None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:51.320300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ac478-eb15-4670-8b61-9d82632bf4aa None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:51.320780 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ac478-eb15-4670-8b61-9d82632bf4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:51.359026 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-7e7ac478-eb15-4670-8b61-9d82632bf4aa None None] MFA Rules not processed for user `5007a2f8e6244b0788f0a554cc572fe5`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:51.362269 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7e7ac478-eb15-4670-8b61-9d82632bf4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:51.384082 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1066/2159] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 14:10:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:51.413226 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707516f-4cff-4840-ad29-4f96be239ca8 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:51.413984 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707516f-4cff-4840-ad29-4f96be239ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:51.415440 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707516f-4cff-4840-ad29-4f96be239ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:51.436000 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ac478-eb15-4670-8b61-9d82632bf4aa None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:51.457352 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ac478-eb15-4670-8b61-9d82632bf4aa None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:51.469829 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-5707516f-4cff-4840-ad29-4f96be239ca8 None None] MFA Rules not processed for user `ebe4b3455a3d4139a1267da97bd83662`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:51.473364 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee16a6-6560-4c77-a5a0-2a9f366d385e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2a6d7bd10435ba2c8e552d808b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:51.474228 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee16a6-6560-4c77-a5a0-2a9f366d385e service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:51.474677 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee16a6-6560-4c77-a5a0-2a9f366d385e service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:51.475508 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee16a6-6560-4c77-a5a0-2a9f366d385e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:51.476233 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-5707516f-4cff-4840-ad29-4f96be239ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:51.492587 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1094/2160] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:10:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:51.499063 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ee16a6-6560-4c77-a5a0-2a9f366d385e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:51.503680 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ee16a6-6560-4c77-a5a0-2a9f366d385e service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:51.526047 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1067/2161] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 14:10:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:10:52.968354 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab19e6a-5920-4b48-b1c5-e49ce74c9f47 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:52.968867 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab19e6a-5920-4b48-b1c5-e49ce74c9f47 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:52.969282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab19e6a-5920-4b48-b1c5-e49ce74c9f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:53.021705 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-bab19e6a-5920-4b48-b1c5-e49ce74c9f47 None None] MFA Rules not processed for user `5007a2f8e6244b0788f0a554cc572fe5`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:53.026757 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-bab19e6a-5920-4b48-b1c5-e49ce74c9f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:53.061900 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1095/2162] 10.222.0.48 () {58 vars in 967 bytes} [Mon Aug 1 14:10:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:53.096517 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee16a6-6560-4c77-a5a0-2a9f366d385e service swift] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:53.119887 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee16a6-6560-4c77-a5a0-2a9f366d385e service swift] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:53.127975 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418fc07-e01a-421d-9286-88b8ee20af2f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2a6d7bd10435ba2c8e552d808b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:53.129154 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418fc07-e01a-421d-9286-88b8ee20af2f service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:53.129616 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418fc07-e01a-421d-9286-88b8ee20af2f service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:53.130040 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418fc07-e01a-421d-9286-88b8ee20af2f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:53.152057 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2418fc07-e01a-421d-9286-88b8ee20af2f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:53.153812 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2418fc07-e01a-421d-9286-88b8ee20af2f service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:53.175176 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1068/2163] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 14:10:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:10:53.836035 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c363ce-d610-4035-a675-4201da8ec884 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:53.836342 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c363ce-d610-4035-a675-4201da8ec884 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:53.836791 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c363ce-d610-4035-a675-4201da8ec884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:53.881204 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-b8c363ce-d610-4035-a675-4201da8ec884 None None] MFA Rules not processed for user `5007a2f8e6244b0788f0a554cc572fe5`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:53.882882 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-b8c363ce-d610-4035-a675-4201da8ec884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:53.897502 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1096/2164] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 14:10:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:53.952843 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418fc07-e01a-421d-9286-88b8ee20af2f service swift] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:53.977185 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418fc07-e01a-421d-9286-88b8ee20af2f service swift] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:54.003446 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9a769-bfd4-44fb-b82a-f2aba8c9bbc1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2a6d7bd10435ba2c8e552d808b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:54.004113 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9a769-bfd4-44fb-b82a-f2aba8c9bbc1 service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:54.004398 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9a769-bfd4-44fb-b82a-f2aba8c9bbc1 service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:54.011917 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9a769-bfd4-44fb-b82a-f2aba8c9bbc1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:54.041989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f9a769-bfd4-44fb-b82a-f2aba8c9bbc1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:54.050424 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f9a769-bfd4-44fb-b82a-f2aba8c9bbc1 service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:54.080976 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1069/2165] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 14:10:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:10:54.958842 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f7634-889e-46fb-ba8e-ef82b9aefb73 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:54.959708 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f7634-889e-46fb-ba8e-ef82b9aefb73 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:54.959708 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f7634-889e-46fb-ba8e-ef82b9aefb73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:54.987710 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-2e9f7634-889e-46fb-ba8e-ef82b9aefb73 None None] MFA Rules not processed for user `5007a2f8e6244b0788f0a554cc572fe5`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:54.989401 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-2e9f7634-889e-46fb-ba8e-ef82b9aefb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:54.998875 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1097/2166] 10.222.0.48 () {58 vars in 967 bytes} [Mon Aug 1 14:10:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:10:55.025355 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9a769-bfd4-44fb-b82a-f2aba8c9bbc1 service swift] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:55.042088 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9a769-bfd4-44fb-b82a-f2aba8c9bbc1 service swift] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:55.054246 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63761fce-73b0-4337-a494-4870dfe9cfb2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2a6d7bd10435ba2c8e552d808b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:55.055079 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63761fce-73b0-4337-a494-4870dfe9cfb2 service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:55.055366 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63761fce-73b0-4337-a494-4870dfe9cfb2 service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:55.055648 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63761fce-73b0-4337-a494-4870dfe9cfb2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:55.077692 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63761fce-73b0-4337-a494-4870dfe9cfb2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:55.080729 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63761fce-73b0-4337-a494-4870dfe9cfb2 service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:55.092078 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1070/2167] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 14:10:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:10:57.422817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f7634-889e-46fb-ba8e-ef82b9aefb73 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:57.424256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-2e9f7634-889e-46fb-ba8e-ef82b9aefb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:57.438357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f7634-889e-46fb-ba8e-ef82b9aefb73 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:57.442949 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103c5e9-9420-4f34-8e43-f1f984157762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:57.443645 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103c5e9-9420-4f34-8e43-f1f984157762 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:57.443936 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103c5e9-9420-4f34-8e43-f1f984157762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:57.444220 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103c5e9-9420-4f34-8e43-f1f984157762 admin admin] PATH_INFO: `/v3/users/7f415b2218694e828b0d160dfce5e114` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:57.445488 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9103c5e9-9420-4f34-8e43-f1f984157762 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f415b2218694e828b0d160dfce5e114)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:57.447445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9103c5e9-9420-4f34-8e43-f1f984157762 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:57.504313 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-9103c5e9-9420-4f34-8e43-f1f984157762 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f415b2218694e828b0d160dfce5e114'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:57.505016 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-9103c5e9-9420-4f34-8e43-f1f984157762 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f415b2218694e828b0d160dfce5e114'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:57.517770 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-9103c5e9-9420-4f34-8e43-f1f984157762 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f415b2218694e828b0d160dfce5e114'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:57.526338 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-9103c5e9-9420-4f34-8e43-f1f984157762 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f415b2218694e828b0d160dfce5e114'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:57.527152 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-9103c5e9-9420-4f34-8e43-f1f984157762 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f415b2218694e828b0d160dfce5e114'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:57.551283 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1098/2168] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:10:57 2022] DELETE /identity/v3/users/7f415b2218694e828b0d160dfce5e114 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:57.551396 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5ff9f-1af5-445d-8a29-3dc52c32a8f1 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:57.551396 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5ff9f-1af5-445d-8a29-3dc52c32a8f1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:57.551396 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5ff9f-1af5-445d-8a29-3dc52c32a8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:57.568221 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b9c5ff9f-1af5-445d-8a29-3dc52c32a8f1 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:57.569509 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b9c5ff9f-1af5-445d-8a29-3dc52c32a8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:57.575608 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:57.639286 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1071/2169] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:10:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:57.647707 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103c5e9-9420-4f34-8e43-f1f984157762 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:57.650368 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-9103c5e9-9420-4f34-8e43-f1f984157762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:57.722282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103c5e9-9420-4f34-8e43-f1f984157762 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:57.726636 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35a243-f673-48f6-842f-a04b5cfdeb4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:57.727934 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35a243-f673-48f6-842f-a04b5cfdeb4e service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:57.728418 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35a243-f673-48f6-842f-a04b5cfdeb4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:57.728929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35a243-f673-48f6-842f-a04b5cfdeb4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:57.740732 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a35a243-f673-48f6-842f-a04b5cfdeb4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:57.742283 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a35a243-f673-48f6-842f-a04b5cfdeb4e service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:57.753801 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1099/2170] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:10:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:10:57.963104 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c5ff9f-1af5-445d-8a29-3dc52c32a8f1 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:57.964828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b9c5ff9f-1af5-445d-8a29-3dc52c32a8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:57.977947 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c5ff9f-1af5-445d-8a29-3dc52c32a8f1 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:57.983677 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c4306f-1ac2-4da3-927b-11a022c9e731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:57.984381 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4306f-1ac2-4da3-927b-11a022c9e731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:57.984705 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4306f-1ac2-4da3-927b-11a022c9e731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:57.985241 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4306f-1ac2-4da3-927b-11a022c9e731 admin admin] PATH_INFO: `/v3/projects/d0cc00c61d3e4e59b5790c99c6277d2d` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:57.986786 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c4306f-1ac2-4da3-927b-11a022c9e731 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0cc00c61d3e4e59b5790c99c6277d2d)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:57.988733 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c4306f-1ac2-4da3-927b-11a022c9e731 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:58.054670 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f6c4306f-1ac2-4da3-927b-11a022c9e731 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0cc00c61d3e4e59b5790c99c6277d2d', 'request_id': 'req-f6c4306f-1ac2-4da3-927b-11a022c9e731'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:58.062189 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f6c4306f-1ac2-4da3-927b-11a022c9e731 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0cc00c61d3e4e59b5790c99c6277d2d', 'request_id': 'req-f6c4306f-1ac2-4da3-927b-11a022c9e731'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:58.068969 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f6c4306f-1ac2-4da3-927b-11a022c9e731 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0cc00c61d3e4e59b5790c99c6277d2d', 'request_id': 'req-f6c4306f-1ac2-4da3-927b-11a022c9e731'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:58.087318 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f6c4306f-1ac2-4da3-927b-11a022c9e731 admin admin] The token cache is being invalidate because project d0cc00c61d3e4e59b5790c99c6277d2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:10:58.088012 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f6c4306f-1ac2-4da3-927b-11a022c9e731 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:58.090272 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f6c4306f-1ac2-4da3-927b-11a022c9e731 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:10:58.099009 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1072/2171] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:10:57 2022] DELETE /identity/v3/projects/d0cc00c61d3e4e59b5790c99c6277d2d => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:58.099214 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce4515-d511-4db8-a9d2-65069e5fae47 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:58.099585 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce4515-d511-4db8-a9d2-65069e5fae47 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:58.099941 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce4515-d511-4db8-a9d2-65069e5fae47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:58.120400 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-e2ce4515-d511-4db8-a9d2-65069e5fae47 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:58.121651 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-e2ce4515-d511-4db8-a9d2-65069e5fae47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:58.150618 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:58.203896 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1100/2172] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:10:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:58.211989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b96845-0220-40c0-a73e-6f59dae50721 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:58.212280 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b96845-0220-40c0-a73e-6f59dae50721 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:58.212596 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b96845-0220-40c0-a73e-6f59dae50721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:58.239201 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-78b96845-0220-40c0-a73e-6f59dae50721 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:58.240472 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-78b96845-0220-40c0-a73e-6f59dae50721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:58.250337 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1073/2173] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:10:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:58.253849 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce4515-d511-4db8-a9d2-65069e5fae47 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:58.270306 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce4515-d511-4db8-a9d2-65069e5fae47 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:58.275630 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d156e-7c7e-42da-9108-527e1620dbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:58.276366 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d156e-7c7e-42da-9108-527e1620dbd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:58.276799 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d156e-7c7e-42da-9108-527e1620dbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:58.277190 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d156e-7c7e-42da-9108-527e1620dbd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:58.277996 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9d156e-7c7e-42da-9108-527e1620dbd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:10:58.278403 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9d156e-7c7e-42da-9108-527e1620dbd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:58.280145 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9d156e-7c7e-42da-9108-527e1620dbd6 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:58.299168 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1101/2174] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:10:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:10:58.301009 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b96845-0220-40c0-a73e-6f59dae50721 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:58.313065 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b96845-0220-40c0-a73e-6f59dae50721 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:58.320421 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a2b00-e4d9-4560-998f-8a3f1049cef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:58.321083 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a2b00-e4d9-4560-998f-8a3f1049cef7 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:58.321366 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a2b00-e4d9-4560-998f-8a3f1049cef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:58.321641 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a2b00-e4d9-4560-998f-8a3f1049cef7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:58.322468 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1a2b00-e4d9-4560-998f-8a3f1049cef7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:58.324635 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1a2b00-e4d9-4560-998f-8a3f1049cef7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:58.372830 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1074/2175] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:10:58 2022] POST /identity/v3/projects => generated 383 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:10:58.377477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d156e-7c7e-42da-9108-527e1620dbd6 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:58.388612 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d156e-7c7e-42da-9108-527e1620dbd6 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:58.465334 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e2fb47-d5db-4aea-82e6-67061c9b58d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:58.466672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2fb47-d5db-4aea-82e6-67061c9b58d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:58.467111 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2fb47-d5db-4aea-82e6-67061c9b58d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:58.467541 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2fb47-d5db-4aea-82e6-67061c9b58d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:58.468363 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e2fb47-d5db-4aea-82e6-67061c9b58d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:58.471003 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e2fb47-d5db-4aea-82e6-67061c9b58d2 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:58.496356 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1102/2176] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:10:58 2022] POST /identity/v3/users => generated 292 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:10:58.500708 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a2b00-e4d9-4560-998f-8a3f1049cef7 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:58.514588 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a2b00-e4d9-4560-998f-8a3f1049cef7 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:58.520670 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866c1cc-fc0b-4a33-aa8b-3c636d525ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:58.521361 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866c1cc-fc0b-4a33-aa8b-3c636d525ecd admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:58.521648 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866c1cc-fc0b-4a33-aa8b-3c636d525ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:58.521926 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866c1cc-fc0b-4a33-aa8b-3c636d525ecd admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:58.522668 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6866c1cc-fc0b-4a33-aa8b-3c636d525ecd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:58.524275 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6866c1cc-fc0b-4a33-aa8b-3c636d525ecd admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:58.535669 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1075/2177] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:10:58.539534 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e2fb47-d5db-4aea-82e6-67061c9b58d2 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:58.552775 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e2fb47-d5db-4aea-82e6-67061c9b58d2 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:58.558800 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e0309-c3fd-4293-b677-d98fbb3b87ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:58.559820 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e0309-c3fd-4293-b677-d98fbb3b87ce admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:58.560364 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e0309-c3fd-4293-b677-d98fbb3b87ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:58.560908 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e0309-c3fd-4293-b677-d98fbb3b87ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:58.561808 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663e0309-c3fd-4293-b677-d98fbb3b87ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:58.563648 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663e0309-c3fd-4293-b677-d98fbb3b87ce admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:58.580458 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1103/2178] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:10:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:10:58.584928 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866c1cc-fc0b-4a33-aa8b-3c636d525ecd admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:58.607305 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866c1cc-fc0b-4a33-aa8b-3c636d525ecd admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:58.628734 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7cffdd-74af-44d4-a33c-7dd7a61a68ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:58.629611 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7cffdd-74af-44d4-a33c-7dd7a61a68ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:58.629989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7cffdd-74af-44d4-a33c-7dd7a61a68ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:58.630353 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7cffdd-74af-44d4-a33c-7dd7a61a68ee admin admin] PATH_INFO: `/v3/projects/99a2e6e5b9974029a065bd70058dc8be/users/1d54f339d88a4f8c9ef914497ce3138d/roles/926b9f36c24847c582c84af904250abc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:58.672468 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7cffdd-74af-44d4-a33c-7dd7a61a68ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=99a2e6e5b9974029a065bd70058dc8be, user_id=1d54f339d88a4f8c9ef914497ce3138d, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:58.676313 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7cffdd-74af-44d4-a33c-7dd7a61a68ee admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:58.694719 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1076/2179] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:10:58 2022] PUT /identity/v3/projects/99a2e6e5b9974029a065bd70058dc8be/users/1d54f339d88a4f8c9ef914497ce3138d/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:10:58.699646 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b174edb-887e-44da-8a46-d7be87f24eb7 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:58.699933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b174edb-887e-44da-8a46-d7be87f24eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:58.700209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b174edb-887e-44da-8a46-d7be87f24eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:58.721294 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-7b174edb-887e-44da-8a46-d7be87f24eb7 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:58.722419 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-7b174edb-887e-44da-8a46-d7be87f24eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:58.790561 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1104/2180] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:10:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:10:58.796674 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7cffdd-74af-44d4-a33c-7dd7a61a68ee admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:10:58.798966 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5f7cffdd-74af-44d4-a33c-7dd7a61a68ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:58.845262 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:10:58.876507 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7cffdd-74af-44d4-a33c-7dd7a61a68ee admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:10:58.880316 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fea56c-5859-4571-9447-6c3c451f8c6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:10:58.880971 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fea56c-5859-4571-9447-6c3c451f8c6e service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:58.881258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fea56c-5859-4571-9447-6c3c451f8c6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:58.881564 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fea56c-5859-4571-9447-6c3c451f8c6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:58.890739 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fea56c-5859-4571-9447-6c3c451f8c6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:10:58.892713 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fea56c-5859-4571-9447-6c3c451f8c6e service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:10:58.903246 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1077/2181] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:10:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:10:59.851899 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c983f-90e5-4ba5-9eb2-ac5ef110ddff None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:10:59.852836 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c983f-90e5-4ba5-9eb2-ac5ef110ddff None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:10:59.853342 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c983f-90e5-4ba5-9eb2-ac5ef110ddff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:10:59.895701 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-779c983f-90e5-4ba5-9eb2-ac5ef110ddff None None] MFA Rules not processed for user `1d54f339d88a4f8c9ef914497ce3138d`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:10:59.903095 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-779c983f-90e5-4ba5-9eb2-ac5ef110ddff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:10:59.946153 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:00.031115 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1105/2182] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:10:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:11:00.044355 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fea56c-5859-4571-9447-6c3c451f8c6e service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:00.047012 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-07fea56c-5859-4571-9447-6c3c451f8c6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:00.162082 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fea56c-5859-4571-9447-6c3c451f8c6e service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:00.168392 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d72509e-f6a0-4ba6-9d64-1b0e0ecf9d34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:00.169080 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d72509e-f6a0-4ba6-9d64-1b0e0ecf9d34 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:00.169363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d72509e-f6a0-4ba6-9d64-1b0e0ecf9d34 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:00.169657 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d72509e-f6a0-4ba6-9d64-1b0e0ecf9d34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:00.180298 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d72509e-f6a0-4ba6-9d64-1b0e0ecf9d34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:00.181845 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d72509e-f6a0-4ba6-9d64-1b0e0ecf9d34 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:00.194263 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1078/2183] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:11:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:11:10.023608 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=17 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:10.187758 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=38 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:11.134017 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c284403-0b2f-4d50-80d1-b3b45dd99677 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:11.134546 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c284403-0b2f-4d50-80d1-b3b45dd99677 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:11.134655 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c284403-0b2f-4d50-80d1-b3b45dd99677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:11.153691 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-7c284403-0b2f-4d50-80d1-b3b45dd99677 None None] MFA Rules not processed for user `1d54f339d88a4f8c9ef914497ce3138d`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:11:11.154911 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-7c284403-0b2f-4d50-80d1-b3b45dd99677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:11.162385 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1106/2184] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:11:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:11:21.148514 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=3 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:26.702687 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d72509e-f6a0-4ba6-9d64-1b0e0ecf9d34 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:26.704101 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5d72509e-f6a0-4ba6-9d64-1b0e0ecf9d34 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:26.718272 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d72509e-f6a0-4ba6-9d64-1b0e0ecf9d34 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:26.723005 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9fd136-45a4-453b-bca6-76deaa775579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:26.723643 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fd136-45a4-453b-bca6-76deaa775579 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:26.723928 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fd136-45a4-453b-bca6-76deaa775579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:26.724204 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fd136-45a4-453b-bca6-76deaa775579 admin admin] PATH_INFO: `/v3/users/4dcdf6d6c1fe4137bb0995ae30b1d4f9` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:26.725393 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9fd136-45a4-453b-bca6-76deaa775579 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dcdf6d6c1fe4137bb0995ae30b1d4f9)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:26.727306 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9fd136-45a4-453b-bca6-76deaa775579 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:26.774838 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7e9fd136-45a4-453b-bca6-76deaa775579 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dcdf6d6c1fe4137bb0995ae30b1d4f9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:11:26.781708 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7e9fd136-45a4-453b-bca6-76deaa775579 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dcdf6d6c1fe4137bb0995ae30b1d4f9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:11:26.782393 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7e9fd136-45a4-453b-bca6-76deaa775579 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dcdf6d6c1fe4137bb0995ae30b1d4f9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:11:26.790186 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7e9fd136-45a4-453b-bca6-76deaa775579 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dcdf6d6c1fe4137bb0995ae30b1d4f9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:11:26.790952 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7e9fd136-45a4-453b-bca6-76deaa775579 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dcdf6d6c1fe4137bb0995ae30b1d4f9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:11:26.809911 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1079/2185] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:11:26 2022] DELETE /identity/v3/users/4dcdf6d6c1fe4137bb0995ae30b1d4f9 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:11:26.816051 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a99bbb-b686-460b-a26c-9b23038f29d1 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:26.819295 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a99bbb-b686-460b-a26c-9b23038f29d1 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:26.819949 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a99bbb-b686-460b-a26c-9b23038f29d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:26.843080 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-55a99bbb-b686-460b-a26c-9b23038f29d1 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:11:26.844294 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-55a99bbb-b686-460b-a26c-9b23038f29d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:26.912638 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1107/2186] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:11:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:11:26.923118 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9fd136-45a4-453b-bca6-76deaa775579 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:26.924556 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7e9fd136-45a4-453b-bca6-76deaa775579 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:26.984652 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9fd136-45a4-453b-bca6-76deaa775579 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:26.988351 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda59ebf-1e19-4a50-8696-98dd3d0a4f45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:26.989014 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda59ebf-1e19-4a50-8696-98dd3d0a4f45 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:26.989294 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda59ebf-1e19-4a50-8696-98dd3d0a4f45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:26.989569 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda59ebf-1e19-4a50-8696-98dd3d0a4f45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:26.999078 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda59ebf-1e19-4a50-8696-98dd3d0a4f45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:27.000567 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda59ebf-1e19-4a50-8696-98dd3d0a4f45 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:27.010542 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1080/2187] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:11:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:11:27.287111 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a99bbb-b686-460b-a26c-9b23038f29d1 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:27.288862 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-55a99bbb-b686-460b-a26c-9b23038f29d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:27.303436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a99bbb-b686-460b-a26c-9b23038f29d1 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:27.309025 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd091950-4af4-48da-a0bc-32a4a314abf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:27.309746 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd091950-4af4-48da-a0bc-32a4a314abf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:27.310038 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd091950-4af4-48da-a0bc-32a4a314abf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:27.310481 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd091950-4af4-48da-a0bc-32a4a314abf5 admin admin] PATH_INFO: `/v3/projects/7808386b690a4255b84d794c761dd024` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:27.311834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd091950-4af4-48da-a0bc-32a4a314abf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7808386b690a4255b84d794c761dd024)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:27.313679 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd091950-4af4-48da-a0bc-32a4a314abf5 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:27.365121 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-fd091950-4af4-48da-a0bc-32a4a314abf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7808386b690a4255b84d794c761dd024', 'request_id': 'req-fd091950-4af4-48da-a0bc-32a4a314abf5'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:11:27.371652 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-fd091950-4af4-48da-a0bc-32a4a314abf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7808386b690a4255b84d794c761dd024', 'request_id': 'req-fd091950-4af4-48da-a0bc-32a4a314abf5'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:11:27.381285 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-fd091950-4af4-48da-a0bc-32a4a314abf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7808386b690a4255b84d794c761dd024', 'request_id': 'req-fd091950-4af4-48da-a0bc-32a4a314abf5'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:11:27.395629 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-fd091950-4af4-48da-a0bc-32a4a314abf5 admin admin] The token cache is being invalidate because project 7808386b690a4255b84d794c761dd024 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:11:27.395916 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-fd091950-4af4-48da-a0bc-32a4a314abf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:11:27.396606 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-fd091950-4af4-48da-a0bc-32a4a314abf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:11:27.398124 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1108/2188] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:11:27 2022] DELETE /identity/v3/projects/7808386b690a4255b84d794c761dd024 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:11:27.403295 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ed01f-05f4-49e7-986a-d76f7ec5187c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:27.403598 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ed01f-05f4-49e7-986a-d76f7ec5187c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:27.403883 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ed01f-05f4-49e7-986a-d76f7ec5187c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:27.425086 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-fe6ed01f-05f4-49e7-986a-d76f7ec5187c None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:11:27.426300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fe6ed01f-05f4-49e7-986a-d76f7ec5187c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:27.491062 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1081/2189] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:11:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:11:27.498662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76332c9e-76f3-4bfb-aba0-123de901917b None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:27.498959 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76332c9e-76f3-4bfb-aba0-123de901917b None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:27.499766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76332c9e-76f3-4bfb-aba0-123de901917b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:27.520581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-76332c9e-76f3-4bfb-aba0-123de901917b None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:11:27.521819 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-76332c9e-76f3-4bfb-aba0-123de901917b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:27.532591 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1109/2190] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:11:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:11:27.537407 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ed01f-05f4-49e7-986a-d76f7ec5187c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:27.555516 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ed01f-05f4-49e7-986a-d76f7ec5187c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:27.560960 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8786464-86b0-4591-87da-380d7a61f8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:27.561592 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8786464-86b0-4591-87da-380d7a61f8f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:27.561912 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8786464-86b0-4591-87da-380d7a61f8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:27.562260 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8786464-86b0-4591-87da-380d7a61f8f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:27.562981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8786464-86b0-4591-87da-380d7a61f8f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:11:27.563291 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8786464-86b0-4591-87da-380d7a61f8f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:27.564925 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8786464-86b0-4591-87da-380d7a61f8f1 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:27.577113 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1082/2191] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:11:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:11:27.582002 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76332c9e-76f3-4bfb-aba0-123de901917b None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:27.592930 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76332c9e-76f3-4bfb-aba0-123de901917b None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:27.599395 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b11fe53-93ec-4663-a7df-bac3fe223dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:27.600039 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11fe53-93ec-4663-a7df-bac3fe223dac admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:27.600467 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11fe53-93ec-4663-a7df-bac3fe223dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:27.600831 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11fe53-93ec-4663-a7df-bac3fe223dac admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:27.601554 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b11fe53-93ec-4663-a7df-bac3fe223dac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:27.603534 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b11fe53-93ec-4663-a7df-bac3fe223dac admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:27.638322 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1110/2192] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:11:27 2022] POST /identity/v3/projects => generated 379 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:11:27.642748 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8786464-86b0-4591-87da-380d7a61f8f1 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:27.647993 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:27.653668 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8786464-86b0-4591-87da-380d7a61f8f1 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:27.720299 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816ff671-78c6-46ad-8722-5f03d8efa967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:27.720970 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ff671-78c6-46ad-8722-5f03d8efa967 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:27.721250 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ff671-78c6-46ad-8722-5f03d8efa967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:27.721522 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ff671-78c6-46ad-8722-5f03d8efa967 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:27.722222 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816ff671-78c6-46ad-8722-5f03d8efa967 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:27.724324 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816ff671-78c6-46ad-8722-5f03d8efa967 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:27.745249 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1083/2193] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:11:27 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:11:27.749584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b11fe53-93ec-4663-a7df-bac3fe223dac admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:27.759830 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b11fe53-93ec-4663-a7df-bac3fe223dac admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:27.764682 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0fe403-5b7f-4692-a2e7-3d103882f6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:27.765332 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0fe403-5b7f-4692-a2e7-3d103882f6fd admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:27.765645 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0fe403-5b7f-4692-a2e7-3d103882f6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:27.765924 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0fe403-5b7f-4692-a2e7-3d103882f6fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:27.766627 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0fe403-5b7f-4692-a2e7-3d103882f6fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:27.768218 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0fe403-5b7f-4692-a2e7-3d103882f6fd admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:27.778051 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1111/2194] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:11:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:11:27.782369 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816ff671-78c6-46ad-8722-5f03d8efa967 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:27.792183 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816ff671-78c6-46ad-8722-5f03d8efa967 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:27.797559 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2294cb-36f6-4920-b959-d191ad242ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:27.798195 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2294cb-36f6-4920-b959-d191ad242ab7 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:27.798474 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2294cb-36f6-4920-b959-d191ad242ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:27.798747 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2294cb-36f6-4920-b959-d191ad242ab7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:27.799441 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2294cb-36f6-4920-b959-d191ad242ab7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:27.801071 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2294cb-36f6-4920-b959-d191ad242ab7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:27.811877 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1084/2195] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:11:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:11:27.816114 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0fe403-5b7f-4692-a2e7-3d103882f6fd admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:27.826342 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0fe403-5b7f-4692-a2e7-3d103882f6fd admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:27.830934 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d5b708-45a5-4e9c-8e47-b99493c5ccde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:27.831589 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5b708-45a5-4e9c-8e47-b99493c5ccde admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:27.831870 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5b708-45a5-4e9c-8e47-b99493c5ccde admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:27.832147 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5b708-45a5-4e9c-8e47-b99493c5ccde admin admin] PATH_INFO: `/v3/projects/7bb93e302f1441fca85409de83230d6a/users/2eb09df64dd74ddeb9fbbf44ae0a3592/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:27.847884 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d5b708-45a5-4e9c-8e47-b99493c5ccde admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bb93e302f1441fca85409de83230d6a, user_id=2eb09df64dd74ddeb9fbbf44ae0a3592, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:27.850553 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d5b708-45a5-4e9c-8e47-b99493c5ccde admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:27.865055 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1112/2196] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:11:27 2022] PUT /identity/v3/projects/7bb93e302f1441fca85409de83230d6a/users/2eb09df64dd74ddeb9fbbf44ae0a3592/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:11:27.870033 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42edce0-8ee1-4cce-bc69-e6e3734dc5f0 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:27.870347 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42edce0-8ee1-4cce-bc69-e6e3734dc5f0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:27.870626 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42edce0-8ee1-4cce-bc69-e6e3734dc5f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:27.889792 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a42edce0-8ee1-4cce-bc69-e6e3734dc5f0 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:11:27.891005 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a42edce0-8ee1-4cce-bc69-e6e3734dc5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:27.950645 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1085/2197] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:11:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:11:27.960345 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d5b708-45a5-4e9c-8e47-b99493c5ccde admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:27.962053 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-88d5b708-45a5-4e9c-8e47-b99493c5ccde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:27.984701 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:28.020578 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d5b708-45a5-4e9c-8e47-b99493c5ccde admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:28.024271 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7367a39-f4b8-426f-8af5-edbfa0059637 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:28.024930 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7367a39-f4b8-426f-8af5-edbfa0059637 service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:28.025215 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7367a39-f4b8-426f-8af5-edbfa0059637 service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:28.025511 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7367a39-f4b8-426f-8af5-edbfa0059637 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:28.035239 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7367a39-f4b8-426f-8af5-edbfa0059637 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:28.036754 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7367a39-f4b8-426f-8af5-edbfa0059637 service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:28.046742 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1113/2198] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:11:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:11:32.874706 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c177d0e-44f7-404e-b9ed-7f60b00c4266 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:32.875306 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c177d0e-44f7-404e-b9ed-7f60b00c4266 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:32.875306 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c177d0e-44f7-404e-b9ed-7f60b00c4266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:32.907714 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-7c177d0e-44f7-404e-b9ed-7f60b00c4266 None None] MFA Rules not processed for user `2eb09df64dd74ddeb9fbbf44ae0a3592`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:11:32.908980 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7c177d0e-44f7-404e-b9ed-7f60b00c4266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:32.994759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1086/2199] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:11:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:11:33.000162 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7367a39-f4b8-426f-8af5-edbfa0059637 service neutron] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:33.010995 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7367a39-f4b8-426f-8af5-edbfa0059637 service neutron] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:33.018012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9c214f-da44-415a-ab7e-13022a2fb3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:33.018799 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c214f-da44-415a-ab7e-13022a2fb3e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:33.019203 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c214f-da44-415a-ab7e-13022a2fb3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:33.019582 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c214f-da44-415a-ab7e-13022a2fb3e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:33.020377 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9c214f-da44-415a-ab7e-13022a2fb3e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:33.022490 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9c214f-da44-415a-ab7e-13022a2fb3e9 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:33.062881 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1114/2200] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:11:32 2022] POST /identity/v3/projects => generated 377 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:11:33.066997 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c177d0e-44f7-404e-b9ed-7f60b00c4266 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:33.082392 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c177d0e-44f7-404e-b9ed-7f60b00c4266 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:33.093494 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:33.156905 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f5bfb-6c31-4387-84b3-1a2875c5a655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:33.157539 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f5bfb-6c31-4387-84b3-1a2875c5a655 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:33.157826 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f5bfb-6c31-4387-84b3-1a2875c5a655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:33.158106 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f5bfb-6c31-4387-84b3-1a2875c5a655 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:33.158777 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8f5bfb-6c31-4387-84b3-1a2875c5a655 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:33.160734 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8f5bfb-6c31-4387-84b3-1a2875c5a655 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:33.185946 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1087/2201] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:11:33 2022] POST /identity/v3/users => generated 295 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:11:33.190487 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9c214f-da44-415a-ab7e-13022a2fb3e9 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:33.204675 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9c214f-da44-415a-ab7e-13022a2fb3e9 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:33.214105 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd71ee-7e7d-4f09-a395-c1d453d646e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:33.215139 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd71ee-7e7d-4f09-a395-c1d453d646e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:33.215317 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd71ee-7e7d-4f09-a395-c1d453d646e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:33.223983 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd71ee-7e7d-4f09-a395-c1d453d646e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:33.225165 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd71ee-7e7d-4f09-a395-c1d453d646e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:33.227069 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd71ee-7e7d-4f09-a395-c1d453d646e5 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:33.246740 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1115/2202] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:11:33.251745 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f5bfb-6c31-4387-84b3-1a2875c5a655 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:33.263166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f5bfb-6c31-4387-84b3-1a2875c5a655 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:33.271934 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7079806e-efda-44c1-b776-24439f659891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:33.272685 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079806e-efda-44c1-b776-24439f659891 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:33.272979 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079806e-efda-44c1-b776-24439f659891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:33.273262 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079806e-efda-44c1-b776-24439f659891 admin admin] PATH_INFO: `/v3/projects/36e85992ca424ff6916434fd4f71c92d/users/73ece04e4ef041f7a949ecb541b748a6/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:33.289242 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7079806e-efda-44c1-b776-24439f659891 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36e85992ca424ff6916434fd4f71c92d, user_id=73ece04e4ef041f7a949ecb541b748a6, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:33.292144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7079806e-efda-44c1-b776-24439f659891 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:33.308015 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1088/2203] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:11:33 2022] PUT /identity/v3/projects/36e85992ca424ff6916434fd4f71c92d/users/73ece04e4ef041f7a949ecb541b748a6/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:11:33.312819 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd71ee-7e7d-4f09-a395-c1d453d646e5 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:33.323347 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd71ee-7e7d-4f09-a395-c1d453d646e5 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:33.384579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358519e-8d74-4b1d-9d7a-96773b0ff361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:33.385211 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358519e-8d74-4b1d-9d7a-96773b0ff361 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:33.385492 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358519e-8d74-4b1d-9d7a-96773b0ff361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:33.385768 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358519e-8d74-4b1d-9d7a-96773b0ff361 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:33.386476 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358519e-8d74-4b1d-9d7a-96773b0ff361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:33.388070 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358519e-8d74-4b1d-9d7a-96773b0ff361 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:33.398630 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1116/2204] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:11:33.402836 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7079806e-efda-44c1-b776-24439f659891 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:33.413603 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7079806e-efda-44c1-b776-24439f659891 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:33.419457 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3febde-6ad9-4876-86cb-1f7bad3ed8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:33.420289 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3febde-6ad9-4876-86cb-1f7bad3ed8ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:33.421236 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3febde-6ad9-4876-86cb-1f7bad3ed8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:33.421658 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3febde-6ad9-4876-86cb-1f7bad3ed8ab admin admin] PATH_INFO: `/v3/projects/36e85992ca424ff6916434fd4f71c92d/users/73ece04e4ef041f7a949ecb541b748a6/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:33.423658 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3febde-6ad9-4876-86cb-1f7bad3ed8ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=36e85992ca424ff6916434fd4f71c92d, user_id=73ece04e4ef041f7a949ecb541b748a6, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:33.426327 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3febde-6ad9-4876-86cb-1f7bad3ed8ab admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:33.438209 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1089/2205] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:11:33 2022] PUT /identity/v3/projects/36e85992ca424ff6916434fd4f71c92d/users/73ece04e4ef041f7a949ecb541b748a6/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:11:39.571977 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c6883-82c7-4507-a3e8-1bf4ca7f4e22 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:39.572581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c6883-82c7-4507-a3e8-1bf4ca7f4e22 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:39.572581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c6883-82c7-4507-a3e8-1bf4ca7f4e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:39.603263 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-ed1c6883-82c7-4507-a3e8-1bf4ca7f4e22 None None] MFA Rules not processed for user `73ece04e4ef041f7a949ecb541b748a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:11:39.604537 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-ed1c6883-82c7-4507-a3e8-1bf4ca7f4e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:39.681349 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1117/2206] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:11:39 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:11:39.695666 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3febde-6ad9-4876-86cb-1f7bad3ed8ab admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:39.697447 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8d3febde-6ad9-4876-86cb-1f7bad3ed8ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:39.777271 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3febde-6ad9-4876-86cb-1f7bad3ed8ab admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:39.783358 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1e93a-a66a-4c07-a0db-a720cc7609aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:39.783993 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1e93a-a66a-4c07-a0db-a720cc7609aa service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:39.784285 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1e93a-a66a-4c07-a0db-a720cc7609aa service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:39.784595 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1e93a-a66a-4c07-a0db-a720cc7609aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:39.852261 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d1e93a-a66a-4c07-a0db-a720cc7609aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:39.853920 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d1e93a-a66a-4c07-a0db-a720cc7609aa service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:39.865233 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:39.865579 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1090/2207] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:11:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:11:40.855613 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1c6883-82c7-4507-a3e8-1bf4ca7f4e22 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:11:40.857302 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-ed1c6883-82c7-4507-a3e8-1bf4ca7f4e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:40.873166 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:40.937712 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1c6883-82c7-4507-a3e8-1bf4ca7f4e22 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:11:40.946409 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a453db0-6caf-4572-b93e-b3bea8d8fd57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:11:40.947255 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a453db0-6caf-4572-b93e-b3bea8d8fd57 service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:40.947744 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a453db0-6caf-4572-b93e-b3bea8d8fd57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:40.948044 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a453db0-6caf-4572-b93e-b3bea8d8fd57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:40.950055 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-5a453db0-6caf-4572-b93e-b3bea8d8fd57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:41.039292 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a453db0-6caf-4572-b93e-b3bea8d8fd57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:11:41.041130 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a453db0-6caf-4572-b93e-b3bea8d8fd57 service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:11:41.053410 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1118/2208] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:11:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:11:45.233656 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef7389-96f8-414a-a51c-464828e59bad None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:11:45.234683 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef7389-96f8-414a-a51c-464828e59bad None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:11:45.234968 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef7389-96f8-414a-a51c-464828e59bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:11:45.282670 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a2ef7389-96f8-414a-a51c-464828e59bad None None] MFA Rules not processed for user `f379b005418a4e399c3b72cbbef6e00d`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:11:45.284394 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a2ef7389-96f8-414a-a51c-464828e59bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:11:45.404323 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1091/2209] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:11:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:11:51.048244 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=45 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:11:55.391305 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=25 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:12.941586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a72a6e-b552-4f24-b27b-8bba52083323 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:12.942113 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a72a6e-b552-4f24-b27b-8bba52083323 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:12.942214 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a72a6e-b552-4f24-b27b-8bba52083323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:12.963355 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-74a72a6e-b552-4f24-b27b-8bba52083323 None None] MFA Rules not processed for user `2eb09df64dd74ddeb9fbbf44ae0a3592`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:12:12.964612 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-74a72a6e-b552-4f24-b27b-8bba52083323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:12.972647 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1119/2210] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:12:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:12:14.261707 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef7389-96f8-414a-a51c-464828e59bad None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:12:14.263729 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a2ef7389-96f8-414a-a51c-464828e59bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:14.333519 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef7389-96f8-414a-a51c-464828e59bad None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:12:14.338919 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aaf995-1891-4b8f-8368-bbe61582d238 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e294fe796a1a4ac89041a532856ad627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:12:14.339542 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aaf995-1891-4b8f-8368-bbe61582d238 service placement] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:14.339835 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aaf995-1891-4b8f-8368-bbe61582d238 service placement] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:14.340129 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aaf995-1891-4b8f-8368-bbe61582d238 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:14.341948 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-79aaf995-1891-4b8f-8368-bbe61582d238 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:14.353017 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aaf995-1891-4b8f-8368-bbe61582d238 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:12:14.354595 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aaf995-1891-4b8f-8368-bbe61582d238 service placement] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:12:14.364321 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1092/2211] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:12:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:12:22.957144 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=3 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:24.360155 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=32 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:45.400179 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e55f4-3007-4990-ba59-6403fdeb3be5 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:45.400745 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e55f4-3007-4990-ba59-6403fdeb3be5 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:45.400849 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e55f4-3007-4990-ba59-6403fdeb3be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:45.421670 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-5f8e55f4-3007-4990-ba59-6403fdeb3be5 None None] MFA Rules not processed for user `ca3cd1c2aec94b43aff0034b812a069d`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:12:45.423188 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-5f8e55f4-3007-4990-ba59-6403fdeb3be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:45.493007 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1120/2212] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:12:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:12:47.812059 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aaf995-1891-4b8f-8368-bbe61582d238 service placement] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:12:47.822631 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aaf995-1891-4b8f-8368-bbe61582d238 service placement] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:12:47.826798 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7674bc14-bc3b-481d-9acb-efa099343d83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:12:47.827418 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674bc14-bc3b-481d-9acb-efa099343d83 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:47.827702 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674bc14-bc3b-481d-9acb-efa099343d83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:47.827981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674bc14-bc3b-481d-9acb-efa099343d83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:47.839566 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7674bc14-bc3b-481d-9acb-efa099343d83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:12:47.841062 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7674bc14-bc3b-481d-9acb-efa099343d83 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:12:47.854680 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1093/2213] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:12:47 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:12:48.242239 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff1b33-baff-425e-b2b3-30ad7b57296f None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:48.242723 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff1b33-baff-425e-b2b3-30ad7b57296f None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:48.242823 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff1b33-baff-425e-b2b3-30ad7b57296f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:48.264527 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-d1ff1b33-baff-425e-b2b3-30ad7b57296f None None] MFA Rules not processed for user `5007a2f8e6244b0788f0a554cc572fe5`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:12:48.265757 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-d1ff1b33-baff-425e-b2b3-30ad7b57296f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:48.334461 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1121/2214] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 14:12:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:12:48.382939 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7674bc14-bc3b-481d-9acb-efa099343d83 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:12:48.384387 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7674bc14-bc3b-481d-9acb-efa099343d83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:12:48.463485 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7674bc14-bc3b-481d-9acb-efa099343d83 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:12:48.468507 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4bade6-c832-4fbb-901a-868ac7a95f21 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2a6d7bd10435ba2c8e552d808b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:12:48.469158 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bade6-c832-4fbb-901a-868ac7a95f21 service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:12:48.469445 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bade6-c832-4fbb-901a-868ac7a95f21 service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:12:48.469724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bade6-c832-4fbb-901a-868ac7a95f21 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:12:48.478970 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4bade6-c832-4fbb-901a-868ac7a95f21 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:12:48.480630 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4bade6-c832-4fbb-901a-868ac7a95f21 service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:12:48.489399 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1094/2215] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 14:12:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 01 14:12:58.327579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=50 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:12:58.485803 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=38 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:02.868762 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf5c85-1f50-4416-bceb-472fe716762a None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:02.869331 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf5c85-1f50-4416-bceb-472fe716762a None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:02.869331 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf5c85-1f50-4416-bceb-472fe716762a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:02.887751 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-0fdf5c85-1f50-4416-bceb-472fe716762a None None] MFA Rules not processed for user `1d54f339d88a4f8c9ef914497ce3138d`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:13:02.888990 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-0fdf5c85-1f50-4416-bceb-472fe716762a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:02.944340 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1122/2216] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:13:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:13:11.922588 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00fd5af-a940-40c5-b13b-e8900f69b4b2 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:11.923133 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00fd5af-a940-40c5-b13b-e8900f69b4b2 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:11.923471 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00fd5af-a940-40c5-b13b-e8900f69b4b2 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:11.925180 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1095/2217] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 14:13:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:13:11.932014 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec2e81-cd6e-4d3e-bb48-2a5117a74c54 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:11.932512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec2e81-cd6e-4d3e-bb48-2a5117a74c54 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:11.932933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec2e81-cd6e-4d3e-bb48-2a5117a74c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:11.953855 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-40ec2e81-cd6e-4d3e-bb48-2a5117a74c54 None None] MFA Rules not processed for user `a246654c783b4cc68d212e91bb05d8c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:13:11.955319 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-40ec2e81-cd6e-4d3e-bb48-2a5117a74c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:11.966354 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1123/2218] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 14:13:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:13:11.992275 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00fd5af-a940-40c5-b13b-e8900f69b4b2 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:11.994648 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b00fd5af-a940-40c5-b13b-e8900f69b4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:12.093580 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00fd5af-a940-40c5-b13b-e8900f69b4b2 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:12.097222 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac4c75-e79e-49d7-80ec-3859c5734ee1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39a66f7362964389be039937a786bb0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:12.097850 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac4c75-e79e-49d7-80ec-3859c5734ee1 service glance] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:12.098135 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac4c75-e79e-49d7-80ec-3859c5734ee1 service glance] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:12.098412 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac4c75-e79e-49d7-80ec-3859c5734ee1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:12.109787 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdac4c75-e79e-49d7-80ec-3859c5734ee1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:12.111228 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdac4c75-e79e-49d7-80ec-3859c5734ee1 service glance] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:12.123110 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1096/2219] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:13:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:13:12.198550 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8a0d0-ebd2-4f98-95ad-fcd45c52d497 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:12.198843 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8a0d0-ebd2-4f98-95ad-fcd45c52d497 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:12.199158 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8a0d0-ebd2-4f98-95ad-fcd45c52d497 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:12.200553 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1124/2220] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 14:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:13:12.205383 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a201ed-34ac-4f19-ba90-25d43ba53d28 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:12.205677 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a201ed-34ac-4f19-ba90-25d43ba53d28 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:12.205959 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a201ed-34ac-4f19-ba90-25d43ba53d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:12.226903 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-51a201ed-34ac-4f19-ba90-25d43ba53d28 None None] MFA Rules not processed for user `fb9f698e09d246bcbea855613764fbe1`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:13:12.228119 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-51a201ed-34ac-4f19-ba90-25d43ba53d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:12.234702 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1097/2221] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 14:13:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:13:12.247318 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8a0d0-ebd2-4f98-95ad-fcd45c52d497 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:12.259034 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8a0d0-ebd2-4f98-95ad-fcd45c52d497 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:12.262695 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1f6adf-179e-4cd7-89cb-42932313ac7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:12.263393 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f6adf-179e-4cd7-89cb-42932313ac7d service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:12.263768 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f6adf-179e-4cd7-89cb-42932313ac7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:12.264058 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f6adf-179e-4cd7-89cb-42932313ac7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:12.271422 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1f6adf-179e-4cd7-89cb-42932313ac7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:12.273590 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1f6adf-179e-4cd7-89cb-42932313ac7d service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:12.281806 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1125/2222] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:13:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:13:12.369769 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6aa6c2-dab0-4456-b1dc-ff9a1e713395 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:12.370160 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6aa6c2-dab0-4456-b1dc-ff9a1e713395 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:12.370506 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6aa6c2-dab0-4456-b1dc-ff9a1e713395 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:12.371952 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1098/2223] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 14:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:13:12.377276 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fbb7e4-f1e4-4bf2-bcfc-4021971bff69 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:12.377635 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fbb7e4-f1e4-4bf2-bcfc-4021971bff69 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:12.378039 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fbb7e4-f1e4-4bf2-bcfc-4021971bff69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:12.397684 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-d4fbb7e4-f1e4-4bf2-bcfc-4021971bff69 None None] MFA Rules not processed for user `a246654c783b4cc68d212e91bb05d8c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:13:12.399578 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-d4fbb7e4-f1e4-4bf2-bcfc-4021971bff69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:12.409023 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1126/2224] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 14:13:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:13:12.419949 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6aa6c2-dab0-4456-b1dc-ff9a1e713395 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:12.434090 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6aa6c2-dab0-4456-b1dc-ff9a1e713395 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:12.437890 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d44acde-7f09-4a1a-bf2b-f31630148ddc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:12.438538 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44acde-7f09-4a1a-bf2b-f31630148ddc service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:12.438819 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44acde-7f09-4a1a-bf2b-f31630148ddc service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:12.439096 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44acde-7f09-4a1a-bf2b-f31630148ddc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:12.450565 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d44acde-7f09-4a1a-bf2b-f31630148ddc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:12.452141 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d44acde-7f09-4a1a-bf2b-f31630148ddc service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:12.463579 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1099/2225] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:13:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:13:20.000749 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d86dd9-8364-4aca-a091-8286387fd65f None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:20.001207 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d86dd9-8364-4aca-a091-8286387fd65f None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:20.001309 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d86dd9-8364-4aca-a091-8286387fd65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:20.019927 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-39d86dd9-8364-4aca-a091-8286387fd65f None None] MFA Rules not processed for user `5007a2f8e6244b0788f0a554cc572fe5`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:13:20.021156 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-39d86dd9-8364-4aca-a091-8286387fd65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:20.028133 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1127/2226] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 14:13:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:13:20.040384 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d44acde-7f09-4a1a-bf2b-f31630148ddc service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:20.051277 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d44acde-7f09-4a1a-bf2b-f31630148ddc service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:20.056920 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c67ec-940f-41c2-9e21-24c3531fa8e0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2a6d7bd10435ba2c8e552d808b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:20.057518 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c67ec-940f-41c2-9e21-24c3531fa8e0 service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:20.057798 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c67ec-940f-41c2-9e21-24c3531fa8e0 service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:20.058097 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c67ec-940f-41c2-9e21-24c3531fa8e0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:20.065885 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360c67ec-940f-41c2-9e21-24c3531fa8e0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:20.067445 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360c67ec-940f-41c2-9e21-24c3531fa8e0 service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:20.075315 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1100/2227] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 14:13:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:13:30.014862 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=40 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:30.072219 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=41 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:13:53.417843 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d86dd9-8364-4aca-a091-8286387fd65f None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:53.427082 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d86dd9-8364-4aca-a091-8286387fd65f None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:53.430980 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467966cc-a734-4f8e-b5f5-89ba126f7157 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:53.431828 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467966cc-a734-4f8e-b5f5-89ba126f7157 service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:53.432164 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467966cc-a734-4f8e-b5f5-89ba126f7157 service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:53.432475 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467966cc-a734-4f8e-b5f5-89ba126f7157 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:53.434281 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-467966cc-a734-4f8e-b5f5-89ba126f7157 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:53.448084 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467966cc-a734-4f8e-b5f5-89ba126f7157 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:53.449542 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467966cc-a734-4f8e-b5f5-89ba126f7157 service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:53.461106 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1128/2228] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:13:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:13:56.995354 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c67ec-940f-41c2-9e21-24c3531fa8e0 service swift] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:13:57.004567 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c67ec-940f-41c2-9e21-24c3531fa8e0 service swift] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:13:57.008133 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1880c-fe34-4be9-a215-5312c268745b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:13:57.008780 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1880c-fe34-4be9-a215-5312c268745b service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:13:57.009066 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1880c-fe34-4be9-a215-5312c268745b service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:13:57.009344 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1880c-fe34-4be9-a215-5312c268745b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:13:57.011064 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f9b1880c-fe34-4be9-a215-5312c268745b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:13:57.024329 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b1880c-fe34-4be9-a215-5312c268745b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:13:57.025840 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b1880c-fe34-4be9-a215-5312c268745b service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:13:57.037096 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1101/2229] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:13:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:14:03.455644 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=6 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:14:06.729768 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f5c7bc-2496-492a-a93b-22b42c8c3a6d None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:06.730347 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f5c7bc-2496-492a-a93b-22b42c8c3a6d None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:06.730347 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f5c7bc-2496-492a-a93b-22b42c8c3a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:06.749430 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-14f5c7bc-2496-492a-a93b-22b42c8c3a6d None None] MFA Rules not processed for user `5007a2f8e6244b0788f0a554cc572fe5`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:14:06.750658 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-14f5c7bc-2496-492a-a93b-22b42c8c3a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:06.757886 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1129/2230] 10.222.0.48 () {58 vars in 967 bytes} [Mon Aug 1 14:14:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:14:06.784226 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1880c-fe34-4be9-a215-5312c268745b service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:06.796368 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1880c-fe34-4be9-a215-5312c268745b service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:06.804612 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c9e637-cac1-4cda-afbb-2b6f90094fbd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2a6d7bd10435ba2c8e552d808b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:06.805248 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9e637-cac1-4cda-afbb-2b6f90094fbd service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:06.805535 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9e637-cac1-4cda-afbb-2b6f90094fbd service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:06.805945 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9e637-cac1-4cda-afbb-2b6f90094fbd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:06.814719 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c9e637-cac1-4cda-afbb-2b6f90094fbd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:06.816516 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c9e637-cac1-4cda-afbb-2b6f90094fbd service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:06.825011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1102/2231] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 14:14:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:14:16.744561 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=3 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:14:16.821706 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=12 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:14:42.008923 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b654a25-a57b-4638-a669-a06f752cde40 None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:42.009436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b654a25-a57b-4638-a669-a06f752cde40 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:42.009537 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b654a25-a57b-4638-a669-a06f752cde40 None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:42.010846 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1130/2232] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 14:14:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:14:42.036213 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbccde36-3619-4748-989f-3af05c471f0b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:42.036808 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbccde36-3619-4748-989f-3af05c471f0b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:42.036808 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbccde36-3619-4748-989f-3af05c471f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:42.057048 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-bbccde36-3619-4748-989f-3af05c471f0b None None] MFA Rules not processed for user `e294fe796a1a4ac89041a532856ad627`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:14:42.058348 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-bbccde36-3619-4748-989f-3af05c471f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:42.069813 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1103/2233] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 14:14:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:14:42.073102 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b654a25-a57b-4638-a669-a06f752cde40 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:42.074879 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-0b654a25-a57b-4638-a669-a06f752cde40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:42.090380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbccde36-3619-4748-989f-3af05c471f0b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:42.101674 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbccde36-3619-4748-989f-3af05c471f0b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:42.106670 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525bac86-7366-4761-8949-2a66a709305c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e294fe796a1a4ac89041a532856ad627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:42.107283 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bac86-7366-4761-8949-2a66a709305c service placement] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:42.107565 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bac86-7366-4761-8949-2a66a709305c service placement] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:42.107838 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bac86-7366-4761-8949-2a66a709305c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:42.116721 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525bac86-7366-4761-8949-2a66a709305c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:42.118293 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525bac86-7366-4761-8949-2a66a709305c service placement] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:42.128213 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1104/2234] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:14:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:14:42.148944 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b654a25-a57b-4638-a669-a06f752cde40 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:42.153416 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96706bc-4890-4130-b036-f17b344c9d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:42.154111 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96706bc-4890-4130-b036-f17b344c9d9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:42.154396 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96706bc-4890-4130-b036-f17b344c9d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:42.154668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96706bc-4890-4130-b036-f17b344c9d9e admin admin] PATH_INFO: `/v3/users/1d54f339d88a4f8c9ef914497ce3138d` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:42.155841 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96706bc-4890-4130-b036-f17b344c9d9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d54f339d88a4f8c9ef914497ce3138d)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:42.157836 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96706bc-4890-4130-b036-f17b344c9d9e admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:42.204921 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d96706bc-4890-4130-b036-f17b344c9d9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d54f339d88a4f8c9ef914497ce3138d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:42.205619 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d96706bc-4890-4130-b036-f17b344c9d9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d54f339d88a4f8c9ef914497ce3138d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:42.217208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d96706bc-4890-4130-b036-f17b344c9d9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d54f339d88a4f8c9ef914497ce3138d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:42.224954 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d96706bc-4890-4130-b036-f17b344c9d9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d54f339d88a4f8c9ef914497ce3138d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:42.225578 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d96706bc-4890-4130-b036-f17b344c9d9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d54f339d88a4f8c9ef914497ce3138d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:42.238547 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1131/2235] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:14:42 2022] DELETE /identity/v3/users/1d54f339d88a4f8c9ef914497ce3138d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:14:42.405797 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525bac86-7366-4761-8949-2a66a709305c service placement] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:42.416790 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525bac86-7366-4761-8949-2a66a709305c service placement] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:42.488013 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d971ad-5337-48a7-a93e-5de440ac5610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:42.488818 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d971ad-5337-48a7-a93e-5de440ac5610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:42.489031 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d971ad-5337-48a7-a93e-5de440ac5610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:42.489306 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d971ad-5337-48a7-a93e-5de440ac5610 admin admin] PATH_INFO: `/v3/projects/99a2e6e5b9974029a065bd70058dc8be` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:42.490464 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d971ad-5337-48a7-a93e-5de440ac5610 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99a2e6e5b9974029a065bd70058dc8be)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:42.492364 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d971ad-5337-48a7-a93e-5de440ac5610 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:42.538460 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-66d971ad-5337-48a7-a93e-5de440ac5610 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99a2e6e5b9974029a065bd70058dc8be', 'request_id': 'req-66d971ad-5337-48a7-a93e-5de440ac5610'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:42.546184 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-66d971ad-5337-48a7-a93e-5de440ac5610 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99a2e6e5b9974029a065bd70058dc8be', 'request_id': 'req-66d971ad-5337-48a7-a93e-5de440ac5610'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:42.552758 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-66d971ad-5337-48a7-a93e-5de440ac5610 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99a2e6e5b9974029a065bd70058dc8be', 'request_id': 'req-66d971ad-5337-48a7-a93e-5de440ac5610'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:42.563293 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-66d971ad-5337-48a7-a93e-5de440ac5610 admin admin] The token cache is being invalidate because project 99a2e6e5b9974029a065bd70058dc8be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:14:42.563575 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-66d971ad-5337-48a7-a93e-5de440ac5610 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:42.564215 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-66d971ad-5337-48a7-a93e-5de440ac5610 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:14:42.565763 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1105/2236] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:14:42 2022] DELETE /identity/v3/projects/99a2e6e5b9974029a065bd70058dc8be => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:14:42.570715 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3a9a9-40ff-4f0a-83af-3af13961ccac None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:42.571014 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3a9a9-40ff-4f0a-83af-3af13961ccac None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:42.571298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3a9a9-40ff-4f0a-83af-3af13961ccac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:42.591311 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-f4e3a9a9-40ff-4f0a-83af-3af13961ccac None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:14:42.592449 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-f4e3a9a9-40ff-4f0a-83af-3af13961ccac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:42.652050 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1132/2237] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:14:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:14:42.659863 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035b34a-5581-4fef-912f-d7df682ced3e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:42.660226 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035b34a-5581-4fef-912f-d7df682ced3e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:42.660619 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035b34a-5581-4fef-912f-d7df682ced3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:42.681089 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8035b34a-5581-4fef-912f-d7df682ced3e None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:14:42.682299 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8035b34a-5581-4fef-912f-d7df682ced3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:42.690802 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1106/2238] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:14:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:14:42.694942 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3a9a9-40ff-4f0a-83af-3af13961ccac None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:42.704796 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3a9a9-40ff-4f0a-83af-3af13961ccac None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:42.709888 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a67dc-6fb6-4e23-8018-9074c86668a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:42.710490 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a67dc-6fb6-4e23-8018-9074c86668a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:42.710769 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a67dc-6fb6-4e23-8018-9074c86668a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:42.711045 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a67dc-6fb6-4e23-8018-9074c86668a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:42.711740 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a67dc-6fb6-4e23-8018-9074c86668a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:14:42.712045 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a67dc-6fb6-4e23-8018-9074c86668a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:42.713662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a67dc-6fb6-4e23-8018-9074c86668a8 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:42.724870 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1133/2239] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:14:42.728980 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035b34a-5581-4fef-912f-d7df682ced3e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:42.738208 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035b34a-5581-4fef-912f-d7df682ced3e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:42.742856 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51a4a7-845a-4e46-be7a-40a338071262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:42.743470 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51a4a7-845a-4e46-be7a-40a338071262 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:42.743757 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51a4a7-845a-4e46-be7a-40a338071262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:42.744037 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51a4a7-845a-4e46-be7a-40a338071262 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:42.744792 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c51a4a7-845a-4e46-be7a-40a338071262 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:42.746698 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c51a4a7-845a-4e46-be7a-40a338071262 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:42.774924 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1107/2240] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:14:42 2022] POST /identity/v3/projects => generated 391 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:14:42.778486 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a67dc-6fb6-4e23-8018-9074c86668a8 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:42.788283 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a67dc-6fb6-4e23-8018-9074c86668a8 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:42.791779 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:14:42.849820 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc605605-60df-405f-8c57-2f8836bb1f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:42.850375 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc605605-60df-405f-8c57-2f8836bb1f89 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:42.850662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc605605-60df-405f-8c57-2f8836bb1f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:42.850943 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc605605-60df-405f-8c57-2f8836bb1f89 admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:42.851622 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc605605-60df-405f-8c57-2f8836bb1f89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:42.853574 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc605605-60df-405f-8c57-2f8836bb1f89 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:42.875064 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1134/2241] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:14:42 2022] POST /identity/v3/users => generated 296 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:14:42.882805 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51a4a7-845a-4e46-be7a-40a338071262 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:42.892980 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51a4a7-845a-4e46-be7a-40a338071262 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:42.897501 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45618275-6f68-4783-a5cc-0126ba839ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:42.898105 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45618275-6f68-4783-a5cc-0126ba839ee0 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:42.898381 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45618275-6f68-4783-a5cc-0126ba839ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:42.898653 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45618275-6f68-4783-a5cc-0126ba839ee0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:42.899340 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45618275-6f68-4783-a5cc-0126ba839ee0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:42.900956 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45618275-6f68-4783-a5cc-0126ba839ee0 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:42.911123 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1108/2242] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:14:42.915050 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc605605-60df-405f-8c57-2f8836bb1f89 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:42.925784 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc605605-60df-405f-8c57-2f8836bb1f89 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:42.930573 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d5e4e-39ce-4486-a32e-221d2533c1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:42.931191 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d5e4e-39ce-4486-a32e-221d2533c1d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:42.931471 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d5e4e-39ce-4486-a32e-221d2533c1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:42.931746 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d5e4e-39ce-4486-a32e-221d2533c1d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:42.932435 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d5e4e-39ce-4486-a32e-221d2533c1d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:42.934074 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d5e4e-39ce-4486-a32e-221d2533c1d0 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:42.945507 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1135/2243] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:14:42.949417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45618275-6f68-4783-a5cc-0126ba839ee0 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:42.959642 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45618275-6f68-4783-a5cc-0126ba839ee0 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:42.964286 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb5be7-1abf-45b2-b898-211d2f1ebab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:42.965050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb5be7-1abf-45b2-b898-211d2f1ebab3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:42.965339 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb5be7-1abf-45b2-b898-211d2f1ebab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:42.965624 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb5be7-1abf-45b2-b898-211d2f1ebab3 admin admin] PATH_INFO: `/v3/projects/227dcdcd97f5432fadda16bfc1b79783/users/c8a20f28482940ffa51be38fdafbfb15/roles/926b9f36c24847c582c84af904250abc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:42.981481 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bb5be7-1abf-45b2-b898-211d2f1ebab3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=227dcdcd97f5432fadda16bfc1b79783, user_id=c8a20f28482940ffa51be38fdafbfb15, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:42.984138 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bb5be7-1abf-45b2-b898-211d2f1ebab3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:42.997349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1109/2244] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:14:42 2022] PUT /identity/v3/projects/227dcdcd97f5432fadda16bfc1b79783/users/c8a20f28482940ffa51be38fdafbfb15/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:14:43.007310 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19fd0e8-c796-4a87-8784-702b865c4df5 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:43.007647 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19fd0e8-c796-4a87-8784-702b865c4df5 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:43.007928 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19fd0e8-c796-4a87-8784-702b865c4df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:43.035836 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-d19fd0e8-c796-4a87-8784-702b865c4df5 None None] MFA Rules not processed for user `c8a20f28482940ffa51be38fdafbfb15`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:14:43.037140 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-d19fd0e8-c796-4a87-8784-702b865c4df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:43.069597 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb5be7-1abf-45b2-b898-211d2f1ebab3 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:43.071277 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-49bb5be7-1abf-45b2-b898-211d2f1ebab3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:43.081939 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:14:43.102205 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1136/2245] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:14:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:14:43.114213 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19fd0e8-c796-4a87-8784-702b865c4df5 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:14:43.115544 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-d19fd0e8-c796-4a87-8784-702b865c4df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:43.148664 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb5be7-1abf-45b2-b898-211d2f1ebab3 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:43.154155 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62dcc4-31e7-437f-a76e-5ae701560af9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:43.154778 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62dcc4-31e7-437f-a76e-5ae701560af9 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:43.155059 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62dcc4-31e7-437f-a76e-5ae701560af9 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:43.155332 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62dcc4-31e7-437f-a76e-5ae701560af9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:43.157129 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4e62dcc4-31e7-437f-a76e-5ae701560af9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:14:43.172739 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19fd0e8-c796-4a87-8784-702b865c4df5 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:14:43.176343 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ad194-44dc-4990-8e41-593176ac77c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:14:43.176980 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ad194-44dc-4990-8e41-593176ac77c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:14:43.177250 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ad194-44dc-4990-8e41-593176ac77c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:14:43.177640 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ad194-44dc-4990-8e41-593176ac77c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:14:43.186409 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605ad194-44dc-4990-8e41-593176ac77c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:43.187842 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605ad194-44dc-4990-8e41-593176ac77c9 service cinder] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:43.197344 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1137/2246] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:14:43 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:14:43.216856 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62dcc4-31e7-437f-a76e-5ae701560af9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:14:43.218865 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62dcc4-31e7-437f-a76e-5ae701560af9 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:14:43.228686 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1110/2247] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:14:43 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:14:53.192785 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:14:53.224872 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=53 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:01.438596 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ad194-44dc-4990-8e41-593176ac77c9 service cinder] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:01.440072 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-605ad194-44dc-4990-8e41-593176ac77c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:01.504050 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ad194-44dc-4990-8e41-593176ac77c9 service cinder] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:01.509436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a98a3e9-743f-4634-8f80-d326528248ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:01.510098 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98a3e9-743f-4634-8f80-d326528248ce service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:01.510343 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98a3e9-743f-4634-8f80-d326528248ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:01.510609 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98a3e9-743f-4634-8f80-d326528248ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:01.512604 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-8a98a3e9-743f-4634-8f80-d326528248ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:01.524105 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a98a3e9-743f-4634-8f80-d326528248ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:01.525877 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a98a3e9-743f-4634-8f80-d326528248ce service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:01.537918 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1138/2248] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:15:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:15:07.711317 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62dcc4-31e7-437f-a76e-5ae701560af9 service nova] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:07.772255 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62dcc4-31e7-437f-a76e-5ae701560af9 service nova] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:07.776758 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebda797-921f-48dd-bc43-683589908d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:07.777412 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebda797-921f-48dd-bc43-683589908d6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:07.777700 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebda797-921f-48dd-bc43-683589908d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:07.777981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebda797-921f-48dd-bc43-683589908d6e admin admin] PATH_INFO: `/v3/users/c8a20f28482940ffa51be38fdafbfb15` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:07.779156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebda797-921f-48dd-bc43-683589908d6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8a20f28482940ffa51be38fdafbfb15)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:07.781121 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebda797-921f-48dd-bc43-683589908d6e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:07.839078 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-eebda797-921f-48dd-bc43-683589908d6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8a20f28482940ffa51be38fdafbfb15'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:07.845994 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-eebda797-921f-48dd-bc43-683589908d6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8a20f28482940ffa51be38fdafbfb15'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:07.846670 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-eebda797-921f-48dd-bc43-683589908d6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8a20f28482940ffa51be38fdafbfb15'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:07.854482 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-eebda797-921f-48dd-bc43-683589908d6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8a20f28482940ffa51be38fdafbfb15'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:07.855442 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-eebda797-921f-48dd-bc43-683589908d6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8a20f28482940ffa51be38fdafbfb15'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:07.875022 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1111/2249] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:15:07 2022] DELETE /identity/v3/users/c8a20f28482940ffa51be38fdafbfb15 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:15:07.880006 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8f54b-618d-4ad1-9b74-e978ff4666a4 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:07.880512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8f54b-618d-4ad1-9b74-e978ff4666a4 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:07.880633 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8f54b-618d-4ad1-9b74-e978ff4666a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:07.899979 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-9dd8f54b-618d-4ad1-9b74-e978ff4666a4 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:07.901225 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-9dd8f54b-618d-4ad1-9b74-e978ff4666a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:07.963766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1139/2250] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:15:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:15:07.973864 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebda797-921f-48dd-bc43-683589908d6e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:07.975384 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-eebda797-921f-48dd-bc43-683589908d6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:08.032342 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebda797-921f-48dd-bc43-683589908d6e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:08.035934 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4a38a-8a63-461c-83a8-39f1614283ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:08.036584 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4a38a-8a63-461c-83a8-39f1614283ce service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:08.036873 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4a38a-8a63-461c-83a8-39f1614283ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:08.037154 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4a38a-8a63-461c-83a8-39f1614283ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:08.046157 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc4a38a-8a63-461c-83a8-39f1614283ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:08.048139 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc4a38a-8a63-461c-83a8-39f1614283ce service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:08.057711 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1112/2251] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:15:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:15:08.342178 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8f54b-618d-4ad1-9b74-e978ff4666a4 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:08.343839 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-9dd8f54b-618d-4ad1-9b74-e978ff4666a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:08.356513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8f54b-618d-4ad1-9b74-e978ff4666a4 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:08.361915 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d98f526-22c6-4ec1-8db5-a6240f4ba07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:08.362565 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98f526-22c6-4ec1-8db5-a6240f4ba07d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:08.362876 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98f526-22c6-4ec1-8db5-a6240f4ba07d admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:08.363180 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98f526-22c6-4ec1-8db5-a6240f4ba07d admin admin] PATH_INFO: `/v3/projects/227dcdcd97f5432fadda16bfc1b79783` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:08.364361 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d98f526-22c6-4ec1-8db5-a6240f4ba07d admin admin] RBAC: Authorizing `identity:delete_project(project_id=227dcdcd97f5432fadda16bfc1b79783)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:08.366354 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d98f526-22c6-4ec1-8db5-a6240f4ba07d admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:08.413252 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-3d98f526-22c6-4ec1-8db5-a6240f4ba07d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '227dcdcd97f5432fadda16bfc1b79783', 'request_id': 'req-3d98f526-22c6-4ec1-8db5-a6240f4ba07d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:08.419370 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-3d98f526-22c6-4ec1-8db5-a6240f4ba07d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '227dcdcd97f5432fadda16bfc1b79783', 'request_id': 'req-3d98f526-22c6-4ec1-8db5-a6240f4ba07d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:08.425490 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-3d98f526-22c6-4ec1-8db5-a6240f4ba07d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '227dcdcd97f5432fadda16bfc1b79783', 'request_id': 'req-3d98f526-22c6-4ec1-8db5-a6240f4ba07d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:08.437520 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-3d98f526-22c6-4ec1-8db5-a6240f4ba07d admin admin] The token cache is being invalidate because project 227dcdcd97f5432fadda16bfc1b79783 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:15:08.437812 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-3d98f526-22c6-4ec1-8db5-a6240f4ba07d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:08.438446 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-3d98f526-22c6-4ec1-8db5-a6240f4ba07d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:15:08.439963 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1140/2252] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:15:08 2022] DELETE /identity/v3/projects/227dcdcd97f5432fadda16bfc1b79783 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:15:08.445171 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a73480-b4d0-403e-99ce-3c14a14be594 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:08.445463 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a73480-b4d0-403e-99ce-3c14a14be594 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:08.445743 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a73480-b4d0-403e-99ce-3c14a14be594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:08.465261 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-67a73480-b4d0-403e-99ce-3c14a14be594 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:08.466458 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-67a73480-b4d0-403e-99ce-3c14a14be594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:08.483836 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:08.527577 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1113/2253] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:15:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:15:08.536381 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef7622-7c55-4d1c-a93a-7529de3284b1 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:08.536716 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef7622-7c55-4d1c-a93a-7529de3284b1 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:08.537005 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef7622-7c55-4d1c-a93a-7529de3284b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:08.556489 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-6fef7622-7c55-4d1c-a93a-7529de3284b1 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:08.557716 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-6fef7622-7c55-4d1c-a93a-7529de3284b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:08.566413 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1141/2254] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:15:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:15:08.570692 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a73480-b4d0-403e-99ce-3c14a14be594 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:08.581281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a73480-b4d0-403e-99ce-3c14a14be594 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:08.585939 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d31b1-3d2f-48f7-84e7-08ab41b602df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:08.586529 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d31b1-3d2f-48f7-84e7-08ab41b602df admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:08.586813 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d31b1-3d2f-48f7-84e7-08ab41b602df admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:08.587089 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d31b1-3d2f-48f7-84e7-08ab41b602df admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:08.587787 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5d31b1-3d2f-48f7-84e7-08ab41b602df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:15:08.588100 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5d31b1-3d2f-48f7-84e7-08ab41b602df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:08.589729 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5d31b1-3d2f-48f7-84e7-08ab41b602df admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:08.602496 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1114/2255] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:15:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:15:08.606895 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef7622-7c55-4d1c-a93a-7529de3284b1 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:08.618198 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef7622-7c55-4d1c-a93a-7529de3284b1 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:08.623534 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2594f6b8-6308-4836-8c68-97722c19865e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:08.624316 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2594f6b8-6308-4836-8c68-97722c19865e admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:08.624644 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2594f6b8-6308-4836-8c68-97722c19865e admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:08.624927 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2594f6b8-6308-4836-8c68-97722c19865e admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:08.625610 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2594f6b8-6308-4836-8c68-97722c19865e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:08.627522 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2594f6b8-6308-4836-8c68-97722c19865e admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:08.650257 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:08.660000 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1142/2256] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:15:08 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:15:08.664042 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d31b1-3d2f-48f7-84e7-08ab41b602df admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:08.673939 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d31b1-3d2f-48f7-84e7-08ab41b602df admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:08.738056 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e87428-e825-4e7e-9a84-b81e08765213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:08.738769 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e87428-e825-4e7e-9a84-b81e08765213 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:08.738981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e87428-e825-4e7e-9a84-b81e08765213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:08.739260 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e87428-e825-4e7e-9a84-b81e08765213 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:08.739927 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e87428-e825-4e7e-9a84-b81e08765213 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:08.741870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e87428-e825-4e7e-9a84-b81e08765213 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:08.763375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1115/2257] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:15:08 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:15:08.767935 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2594f6b8-6308-4836-8c68-97722c19865e admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:08.778095 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2594f6b8-6308-4836-8c68-97722c19865e admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:08.783730 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72a51b-0ef7-4b55-a47c-6d7257a4d8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:08.784366 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72a51b-0ef7-4b55-a47c-6d7257a4d8de admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:08.784680 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72a51b-0ef7-4b55-a47c-6d7257a4d8de admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:08.784965 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72a51b-0ef7-4b55-a47c-6d7257a4d8de admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:08.785671 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac72a51b-0ef7-4b55-a47c-6d7257a4d8de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:08.787333 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac72a51b-0ef7-4b55-a47c-6d7257a4d8de admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:08.798295 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1143/2258] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:15:08.801844 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e87428-e825-4e7e-9a84-b81e08765213 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:08.812132 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e87428-e825-4e7e-9a84-b81e08765213 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:08.819227 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c3934-2a24-4ac7-9fb3-bbccf0bb56ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:08.819412 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c3934-2a24-4ac7-9fb3-bbccf0bb56ef admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:08.819862 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c3934-2a24-4ac7-9fb3-bbccf0bb56ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:08.820256 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c3934-2a24-4ac7-9fb3-bbccf0bb56ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:08.821806 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00c3934-2a24-4ac7-9fb3-bbccf0bb56ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:08.822906 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00c3934-2a24-4ac7-9fb3-bbccf0bb56ef admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:08.836928 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1116/2259] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:15:08.841048 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72a51b-0ef7-4b55-a47c-6d7257a4d8de admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:08.852218 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72a51b-0ef7-4b55-a47c-6d7257a4d8de admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:08.856920 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92583298-5982-4811-9837-4787e22e207f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:08.857584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92583298-5982-4811-9837-4787e22e207f admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:08.857869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92583298-5982-4811-9837-4787e22e207f admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:08.858148 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92583298-5982-4811-9837-4787e22e207f admin admin] PATH_INFO: `/v3/projects/d11b6361110344b0a67b6b73ccc0c2f4/users/86cef19aed2a45fb87c30dbd2d422dae/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:08.873670 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92583298-5982-4811-9837-4787e22e207f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d11b6361110344b0a67b6b73ccc0c2f4, user_id=86cef19aed2a45fb87c30dbd2d422dae, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:08.877125 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92583298-5982-4811-9837-4787e22e207f admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:08.891969 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1144/2260] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:15:08 2022] PUT /identity/v3/projects/d11b6361110344b0a67b6b73ccc0c2f4/users/86cef19aed2a45fb87c30dbd2d422dae/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:15:08.896982 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f469a7-0417-4e61-ba42-837dd38a583c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:08.897279 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f469a7-0417-4e61-ba42-837dd38a583c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:08.897570 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f469a7-0417-4e61-ba42-837dd38a583c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:08.916151 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-82f469a7-0417-4e61-ba42-837dd38a583c None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:08.917407 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-82f469a7-0417-4e61-ba42-837dd38a583c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:08.982040 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1117/2261] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:15:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:15:08.991739 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92583298-5982-4811-9837-4787e22e207f admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:08.993492 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-92583298-5982-4811-9837-4787e22e207f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:09.052501 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92583298-5982-4811-9837-4787e22e207f admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:09.056152 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae520f-8717-481a-b26f-4b2aeec6bbdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:09.056841 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae520f-8717-481a-b26f-4b2aeec6bbdc service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:09.057128 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae520f-8717-481a-b26f-4b2aeec6bbdc service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:09.057537 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae520f-8717-481a-b26f-4b2aeec6bbdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:09.068772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcae520f-8717-481a-b26f-4b2aeec6bbdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:09.070263 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcae520f-8717-481a-b26f-4b2aeec6bbdc service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:09.081199 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1145/2262] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:15:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:15:13.878237 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c4ab4-d096-4efd-a231-d2a7ecf68795 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:13.878742 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c4ab4-d096-4efd-a231-d2a7ecf68795 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:13.878839 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c4ab4-d096-4efd-a231-d2a7ecf68795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:13.907337 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-938c4ab4-d096-4efd-a231-d2a7ecf68795 None None] MFA Rules not processed for user `86cef19aed2a45fb87c30dbd2d422dae`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:13.908579 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-938c4ab4-d096-4efd-a231-d2a7ecf68795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:13.926817 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:14.010747 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1118/2263] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:15:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:15:14.015189 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae520f-8717-481a-b26f-4b2aeec6bbdc service neutron] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:14.025046 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae520f-8717-481a-b26f-4b2aeec6bbdc service neutron] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:14.029598 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167377b0-7ec6-4f14-83b2-b3a5d75851c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:14.030200 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167377b0-7ec6-4f14-83b2-b3a5d75851c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:14.030487 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167377b0-7ec6-4f14-83b2-b3a5d75851c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:14.030766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167377b0-7ec6-4f14-83b2-b3a5d75851c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:14.031436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167377b0-7ec6-4f14-83b2-b3a5d75851c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:14.033376 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167377b0-7ec6-4f14-83b2-b3a5d75851c6 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:14.060268 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1146/2264] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:15:14 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:15:14.065062 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c4ab4-d096-4efd-a231-d2a7ecf68795 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:14.075759 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c4ab4-d096-4efd-a231-d2a7ecf68795 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:14.133222 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c08e9f-ec95-4674-b88c-741d1a4cbd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:14.133848 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c08e9f-ec95-4674-b88c-741d1a4cbd22 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:14.134132 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c08e9f-ec95-4674-b88c-741d1a4cbd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:14.134410 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c08e9f-ec95-4674-b88c-741d1a4cbd22 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:14.135086 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c08e9f-ec95-4674-b88c-741d1a4cbd22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:14.137073 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c08e9f-ec95-4674-b88c-741d1a4cbd22 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:14.158913 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1119/2265] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:15:14 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:15:14.162934 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167377b0-7ec6-4f14-83b2-b3a5d75851c6 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:14.174489 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167377b0-7ec6-4f14-83b2-b3a5d75851c6 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:14.179193 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9931076d-a54c-4d0b-836a-e94e23f7dfe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:14.179814 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931076d-a54c-4d0b-836a-e94e23f7dfe5 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:14.180100 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931076d-a54c-4d0b-836a-e94e23f7dfe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:14.180379 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931076d-a54c-4d0b-836a-e94e23f7dfe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:14.181129 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9931076d-a54c-4d0b-836a-e94e23f7dfe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:14.182834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9931076d-a54c-4d0b-836a-e94e23f7dfe5 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:14.195051 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1147/2266] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:15:14.199262 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c08e9f-ec95-4674-b88c-741d1a4cbd22 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:14.209843 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c08e9f-ec95-4674-b88c-741d1a4cbd22 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:14.214848 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a3c544-9e17-4510-8e8e-f9422d9164c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:14.215558 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3c544-9e17-4510-8e8e-f9422d9164c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:14.215841 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3c544-9e17-4510-8e8e-f9422d9164c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:14.216118 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3c544-9e17-4510-8e8e-f9422d9164c5 admin admin] PATH_INFO: `/v3/projects/0808b80655a349a792b3fb4c96cab884/users/fd6769ee862844e19b53ff6855acd760/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:14.236932 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a3c544-9e17-4510-8e8e-f9422d9164c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0808b80655a349a792b3fb4c96cab884, user_id=fd6769ee862844e19b53ff6855acd760, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:14.240326 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a3c544-9e17-4510-8e8e-f9422d9164c5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:14.259832 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1120/2267] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:15:14 2022] PUT /identity/v3/projects/0808b80655a349a792b3fb4c96cab884/users/fd6769ee862844e19b53ff6855acd760/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:15:14.263624 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9931076d-a54c-4d0b-836a-e94e23f7dfe5 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:14.276389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9931076d-a54c-4d0b-836a-e94e23f7dfe5 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:14.355108 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7eccd0-5c8d-4b43-a327-b4c3a7d61402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:14.355874 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7eccd0-5c8d-4b43-a327-b4c3a7d61402 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:14.356733 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7eccd0-5c8d-4b43-a327-b4c3a7d61402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:14.357175 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7eccd0-5c8d-4b43-a327-b4c3a7d61402 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:14.358590 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7eccd0-5c8d-4b43-a327-b4c3a7d61402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:14.362398 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7eccd0-5c8d-4b43-a327-b4c3a7d61402 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:14.374835 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1148/2268] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:15:14.379108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a3c544-9e17-4510-8e8e-f9422d9164c5 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:14.389872 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a3c544-9e17-4510-8e8e-f9422d9164c5 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:14.395162 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e93e9f-49cc-4fe8-b08f-896246395a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:14.395852 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e93e9f-49cc-4fe8-b08f-896246395a35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:14.396277 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e93e9f-49cc-4fe8-b08f-896246395a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:14.396603 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e93e9f-49cc-4fe8-b08f-896246395a35 admin admin] PATH_INFO: `/v3/projects/0808b80655a349a792b3fb4c96cab884/users/fd6769ee862844e19b53ff6855acd760/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:14.398986 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e93e9f-49cc-4fe8-b08f-896246395a35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0808b80655a349a792b3fb4c96cab884, user_id=fd6769ee862844e19b53ff6855acd760, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:14.401926 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e93e9f-49cc-4fe8-b08f-896246395a35 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:14.416524 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1121/2269] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:15:14 2022] PUT /identity/v3/projects/0808b80655a349a792b3fb4c96cab884/users/fd6769ee862844e19b53ff6855acd760/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:15:18.978771 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4103b6-f125-4c7a-b9ca-544ffe1f1287 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:18.979291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4103b6-f125-4c7a-b9ca-544ffe1f1287 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:18.979291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4103b6-f125-4c7a-b9ca-544ffe1f1287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:19.008108 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-5f4103b6-f125-4c7a-b9ca-544ffe1f1287 None None] MFA Rules not processed for user `fd6769ee862844e19b53ff6855acd760`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:19.009426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-5f4103b6-f125-4c7a-b9ca-544ffe1f1287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:19.026642 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:19.080570 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1149/2270] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:15:18 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:15:19.095167 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e93e9f-49cc-4fe8-b08f-896246395a35 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:19.096935 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-78e93e9f-49cc-4fe8-b08f-896246395a35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:19.176238 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e93e9f-49cc-4fe8-b08f-896246395a35 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:19.181869 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16276dc4-37ec-4351-af23-dbb5151ac3d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:19.182507 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16276dc4-37ec-4351-af23-dbb5151ac3d9 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:19.182800 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16276dc4-37ec-4351-af23-dbb5151ac3d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:19.183076 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16276dc4-37ec-4351-af23-dbb5151ac3d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:19.189423 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:19.255885 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16276dc4-37ec-4351-af23-dbb5151ac3d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:19.255885 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16276dc4-37ec-4351-af23-dbb5151ac3d9 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:19.267900 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1122/2271] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:15:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:15:20.379552 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda17e97-746e-42d4-97e9-7eb893565745 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:20.380048 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda17e97-746e-42d4-97e9-7eb893565745 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:20.380256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda17e97-746e-42d4-97e9-7eb893565745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:20.400743 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-eda17e97-746e-42d4-97e9-7eb893565745 None None] MFA Rules not processed for user `86cef19aed2a45fb87c30dbd2d422dae`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:20.401959 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-eda17e97-746e-42d4-97e9-7eb893565745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:20.409390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1150/2272] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:15:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:15:21.275611 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec04189-c390-45de-b992-aec3ad5863df None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:21.276063 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec04189-c390-45de-b992-aec3ad5863df None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:21.276181 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec04189-c390-45de-b992-aec3ad5863df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:21.296023 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-3ec04189-c390-45de-b992-aec3ad5863df None None] MFA Rules not processed for user `f379b005418a4e399c3b72cbbef6e00d`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:15:21.297269 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3ec04189-c390-45de-b992-aec3ad5863df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:21.361206 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1123/2273] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:15:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:15:27.499156 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda17e97-746e-42d4-97e9-7eb893565745 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:27.502057 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-eda17e97-746e-42d4-97e9-7eb893565745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:27.522156 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda17e97-746e-42d4-97e9-7eb893565745 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:27.527840 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271bc03-b355-40c8-ad1b-21caf4c9a8f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:27.528472 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271bc03-b355-40c8-ad1b-21caf4c9a8f9 service nova] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:27.528779 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271bc03-b355-40c8-ad1b-21caf4c9a8f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:27.529053 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271bc03-b355-40c8-ad1b-21caf4c9a8f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:27.530917 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-7271bc03-b355-40c8-ad1b-21caf4c9a8f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:27.543908 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7271bc03-b355-40c8-ad1b-21caf4c9a8f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:27.545513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7271bc03-b355-40c8-ad1b-21caf4c9a8f9 service nova] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:27.557796 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1151/2274] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:15:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:15:31.353791 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=49 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:37.552079 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=30 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:15:45.176086 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec04189-c390-45de-b992-aec3ad5863df None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:45.191175 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec04189-c390-45de-b992-aec3ad5863df None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:45.196719 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c461b4d-11f4-4527-96c6-54b3a4400543 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:45.197464 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c461b4d-11f4-4527-96c6-54b3a4400543 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:45.197815 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c461b4d-11f4-4527-96c6-54b3a4400543 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:45.198161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c461b4d-11f4-4527-96c6-54b3a4400543 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:45.200044 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4c461b4d-11f4-4527-96c6-54b3a4400543 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:45.258990 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c461b4d-11f4-4527-96c6-54b3a4400543 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:45.260637 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c461b4d-11f4-4527-96c6-54b3a4400543 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:45.269793 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1124/2275] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:15:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:15:45.692212 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271bc03-b355-40c8-ad1b-21caf4c9a8f9 service nova] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:45.747320 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271bc03-b355-40c8-ad1b-21caf4c9a8f9 service nova] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:45.751465 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ec9d51-58c1-43c4-9b85-8ed75ddba5d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:45.752086 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec9d51-58c1-43c4-9b85-8ed75ddba5d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:45.752371 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec9d51-58c1-43c4-9b85-8ed75ddba5d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:45.752679 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec9d51-58c1-43c4-9b85-8ed75ddba5d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:45.754487 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-05ec9d51-58c1-43c4-9b85-8ed75ddba5d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:45.763862 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ec9d51-58c1-43c4-9b85-8ed75ddba5d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:45.765319 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ec9d51-58c1-43c4-9b85-8ed75ddba5d0 service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:45.773800 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1152/2276] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:15:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:15:46.206707 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c461b4d-11f4-4527-96c6-54b3a4400543 service nova] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:46.208151 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4c461b4d-11f4-4527-96c6-54b3a4400543 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:46.278193 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c461b4d-11f4-4527-96c6-54b3a4400543 service nova] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:46.283158 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e8d5a-cde3-49d7-b16a-78bf593fef15 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e294fe796a1a4ac89041a532856ad627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:46.283752 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e8d5a-cde3-49d7-b16a-78bf593fef15 service placement] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:46.284036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e8d5a-cde3-49d7-b16a-78bf593fef15 service placement] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:46.284310 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e8d5a-cde3-49d7-b16a-78bf593fef15 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:46.286085 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9d4e8d5a-cde3-49d7-b16a-78bf593fef15 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:46.297477 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e8d5a-cde3-49d7-b16a-78bf593fef15 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:46.299105 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e8d5a-cde3-49d7-b16a-78bf593fef15 service placement] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:46.308641 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1125/2277] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:15:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:15:49.203115 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ec9d51-58c1-43c4-9b85-8ed75ddba5d0 service neutron] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:49.233833 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ec9d51-58c1-43c4-9b85-8ed75ddba5d0 service neutron] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:49.242279 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f94077-1392-4d2d-b1fd-397ff508d99e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:49.243405 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f94077-1392-4d2d-b1fd-397ff508d99e service nova] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:49.244250 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f94077-1392-4d2d-b1fd-397ff508d99e service nova] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:49.244729 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f94077-1392-4d2d-b1fd-397ff508d99e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:49.248214 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-48f94077-1392-4d2d-b1fd-397ff508d99e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:49.267265 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f94077-1392-4d2d-b1fd-397ff508d99e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:49.269558 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f94077-1392-4d2d-b1fd-397ff508d99e service nova] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:49.282650 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1153/2278] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:15:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:15:55.431297 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e8d5a-cde3-49d7-b16a-78bf593fef15 service placement] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:15:55.443981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e8d5a-cde3-49d7-b16a-78bf593fef15 service placement] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:15:55.449445 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f226f-ce94-4be4-ad58-3d01287cb220 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:15:55.450318 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f226f-ce94-4be4-ad58-3d01287cb220 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:15:55.450603 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f226f-ce94-4be4-ad58-3d01287cb220 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:15:55.450884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f226f-ce94-4be4-ad58-3d01287cb220 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:15:55.452674 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cc2f226f-ce94-4be4-ad58-3d01287cb220 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:15:55.510905 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2f226f-ce94-4be4-ad58-3d01287cb220 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:15:55.512510 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2f226f-ce94-4be4-ad58-3d01287cb220 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:15:55.523325 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1126/2279] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:15:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:15:59.277115 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=32 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:16:03.012208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f94077-1392-4d2d-b1fd-397ff508d99e service nova] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:03.032368 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f94077-1392-4d2d-b1fd-397ff508d99e service nova] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:03.049795 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40d9ce-49c5-4482-adb0-deda02d4120a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:03.050686 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40d9ce-49c5-4482-adb0-deda02d4120a service nova] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:03.052866 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40d9ce-49c5-4482-adb0-deda02d4120a service nova] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:03.053381 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40d9ce-49c5-4482-adb0-deda02d4120a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:03.059341 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-4e40d9ce-49c5-4482-adb0-deda02d4120a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:03.194819 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e40d9ce-49c5-4482-adb0-deda02d4120a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:03.196620 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e40d9ce-49c5-4482-adb0-deda02d4120a service nova] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:03.214173 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1154/2280] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:16:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:16:03.466679 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f226f-ce94-4be4-ad58-3d01287cb220 service nova] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:03.483014 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f226f-ce94-4be4-ad58-3d01287cb220 service nova] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:03.490952 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a60556-7014-41d3-973a-63c391dde048 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:03.493013 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a60556-7014-41d3-973a-63c391dde048 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:03.494336 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a60556-7014-41d3-973a-63c391dde048 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:03.494839 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a60556-7014-41d3-973a-63c391dde048 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:03.497164 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d5a60556-7014-41d3-973a-63c391dde048 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:03.512684 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a60556-7014-41d3-973a-63c391dde048 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:03.515380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a60556-7014-41d3-973a-63c391dde048 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:03.563636 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1127/2281] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:16:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:16:05.187192 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb94cf-6a7f-4b1c-8a19-3d0e3a530f99 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:05.188287 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb94cf-6a7f-4b1c-8a19-3d0e3a530f99 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:05.188804 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb94cf-6a7f-4b1c-8a19-3d0e3a530f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:05.215312 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-e6bb94cf-6a7f-4b1c-8a19-3d0e3a530f99 None None] MFA Rules not processed for user `ca3cd1c2aec94b43aff0034b812a069d`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:16:05.216707 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-e6bb94cf-6a7f-4b1c-8a19-3d0e3a530f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:05.224794 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1155/2282] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:16:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:16:05.988073 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a60556-7014-41d3-973a-63c391dde048 service nova] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:05.989875 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d5a60556-7014-41d3-973a-63c391dde048 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:06.008862 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a60556-7014-41d3-973a-63c391dde048 service nova] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:06.013882 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30506b91-27a2-40b4-8a04-08ba400f6bab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:06.015659 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30506b91-27a2-40b4-8a04-08ba400f6bab service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:06.015963 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30506b91-27a2-40b4-8a04-08ba400f6bab service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:06.016245 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30506b91-27a2-40b4-8a04-08ba400f6bab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:06.018369 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-30506b91-27a2-40b4-8a04-08ba400f6bab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:06.038372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30506b91-27a2-40b4-8a04-08ba400f6bab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:06.040468 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30506b91-27a2-40b4-8a04-08ba400f6bab service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:06.046089 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:16:06.052841 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1128/2283] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:16:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:16:07.221730 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c698d-b019-4033-807b-4726f627bee4 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:07.222235 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c698d-b019-4033-807b-4726f627bee4 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:07.223151 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c698d-b019-4033-807b-4726f627bee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:07.234373 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30506b91-27a2-40b4-8a04-08ba400f6bab service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:07.261417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30506b91-27a2-40b4-8a04-08ba400f6bab service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:07.262755 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-b93c698d-b019-4033-807b-4726f627bee4 None None] MFA Rules not processed for user `62cc969defd64837a20e9a7dbda5b4fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:16:07.264455 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-b93c698d-b019-4033-807b-4726f627bee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:07.267932 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba960b1-99ac-46da-bb48-c963d779a92f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:07.269181 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba960b1-99ac-46da-bb48-c963d779a92f service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:07.269706 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba960b1-99ac-46da-bb48-c963d779a92f service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:07.270169 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba960b1-99ac-46da-bb48-c963d779a92f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:07.274954 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1156/2284] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:16:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:16:07.290816 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba960b1-99ac-46da-bb48-c963d779a92f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:07.292653 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba960b1-99ac-46da-bb48-c963d779a92f service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:07.306989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1129/2285] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:16:07 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:16:14.276765 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f806b-b992-4dfa-9d02-54d1954320f6 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:14.277213 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f806b-b992-4dfa-9d02-54d1954320f6 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:14.277330 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f806b-b992-4dfa-9d02-54d1954320f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:14.295905 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-e57f806b-b992-4dfa-9d02-54d1954320f6 None None] MFA Rules not processed for user `5007a2f8e6244b0788f0a554cc572fe5`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:16:14.297164 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-e57f806b-b992-4dfa-9d02-54d1954320f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:14.350889 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1157/2286] 10.222.0.48 () {58 vars in 967 bytes} [Mon Aug 1 14:16:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:16:14.413647 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba960b1-99ac-46da-bb48-c963d779a92f service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:14.415281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4ba960b1-99ac-46da-bb48-c963d779a92f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:14.488566 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba960b1-99ac-46da-bb48-c963d779a92f service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:14.493525 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c218a2e5-7195-41d0-bd4b-43ee7f100d39 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2a6d7bd10435ba2c8e552d808b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:14.494141 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c218a2e5-7195-41d0-bd4b-43ee7f100d39 service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:14.494425 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c218a2e5-7195-41d0-bd4b-43ee7f100d39 service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:14.494703 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c218a2e5-7195-41d0-bd4b-43ee7f100d39 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:14.503214 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c218a2e5-7195-41d0-bd4b-43ee7f100d39 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:14.504830 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c218a2e5-7195-41d0-bd4b-43ee7f100d39 service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:14.512667 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1130/2287] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 14:16:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:16:14.833307 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f806b-b992-4dfa-9d02-54d1954320f6 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:14.842945 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f806b-b992-4dfa-9d02-54d1954320f6 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:14.847076 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84065bd2-4b6b-449d-9986-2201515efd6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:14.847704 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84065bd2-4b6b-449d-9986-2201515efd6e service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:14.848209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84065bd2-4b6b-449d-9986-2201515efd6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:14.848809 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84065bd2-4b6b-449d-9986-2201515efd6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:14.851429 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-84065bd2-4b6b-449d-9986-2201515efd6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:14.917847 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84065bd2-4b6b-449d-9986-2201515efd6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:14.919291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84065bd2-4b6b-449d-9986-2201515efd6e service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:14.929150 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1158/2288] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:16:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:16:18.467692 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c218a2e5-7195-41d0-bd4b-43ee7f100d39 service swift] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:18.469234 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c218a2e5-7195-41d0-bd4b-43ee7f100d39 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:18.484163 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c218a2e5-7195-41d0-bd4b-43ee7f100d39 service swift] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:18.489141 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61b0ac8-0f7e-42c7-acb7-dcbce5571cdb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e294fe796a1a4ac89041a532856ad627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:18.489758 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b0ac8-0f7e-42c7-acb7-dcbce5571cdb service placement] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:18.490044 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b0ac8-0f7e-42c7-acb7-dcbce5571cdb service placement] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:18.490321 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b0ac8-0f7e-42c7-acb7-dcbce5571cdb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:18.492026 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a61b0ac8-0f7e-42c7-acb7-dcbce5571cdb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:18.505018 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61b0ac8-0f7e-42c7-acb7-dcbce5571cdb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:18.506801 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61b0ac8-0f7e-42c7-acb7-dcbce5571cdb service placement] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:18.520532 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1131/2289] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:16:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:16:23.375089 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84065bd2-4b6b-449d-9986-2201515efd6e service neutron] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:23.376910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-84065bd2-4b6b-449d-9986-2201515efd6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:23.391472 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84065bd2-4b6b-449d-9986-2201515efd6e service neutron] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:23.396156 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f516e-7392-4025-bfd2-01b52dc20b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:23.396868 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f516e-7392-4025-bfd2-01b52dc20b73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:23.397156 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f516e-7392-4025-bfd2-01b52dc20b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:23.397438 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f516e-7392-4025-bfd2-01b52dc20b73 admin admin] PATH_INFO: `/v3/users/b6ed2d0b015d4fe39d780306d04fba4d` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:23.398606 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13f516e-7392-4025-bfd2-01b52dc20b73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6ed2d0b015d4fe39d780306d04fba4d)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:23.400594 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13f516e-7392-4025-bfd2-01b52dc20b73 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:23.424418 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:16:23.454824 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-a13f516e-7392-4025-bfd2-01b52dc20b73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6ed2d0b015d4fe39d780306d04fba4d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:23.455505 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-a13f516e-7392-4025-bfd2-01b52dc20b73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6ed2d0b015d4fe39d780306d04fba4d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:23.467876 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-a13f516e-7392-4025-bfd2-01b52dc20b73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6ed2d0b015d4fe39d780306d04fba4d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:23.475598 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-a13f516e-7392-4025-bfd2-01b52dc20b73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6ed2d0b015d4fe39d780306d04fba4d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:23.476328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-a13f516e-7392-4025-bfd2-01b52dc20b73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6ed2d0b015d4fe39d780306d04fba4d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:23.490099 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1159/2290] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:16:23 2022] DELETE /identity/v3/users/b6ed2d0b015d4fe39d780306d04fba4d => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:16:23.689289 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61b0ac8-0f7e-42c7-acb7-dcbce5571cdb service placement] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:23.704803 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61b0ac8-0f7e-42c7-acb7-dcbce5571cdb service placement] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:23.805259 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d3b30-6a4e-4eaf-95e4-287f58ff1c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:23.806237 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d3b30-6a4e-4eaf-95e4-287f58ff1c71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:23.806733 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d3b30-6a4e-4eaf-95e4-287f58ff1c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:23.807178 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d3b30-6a4e-4eaf-95e4-287f58ff1c71 admin admin] PATH_INFO: `/v3/projects/a8630c7be7ea49d5b6d3ef6d46fd653e` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:23.808942 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d3b30-6a4e-4eaf-95e4-287f58ff1c71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8630c7be7ea49d5b6d3ef6d46fd653e)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:23.811972 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d3b30-6a4e-4eaf-95e4-287f58ff1c71 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:23.867105 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ba8d3b30-6a4e-4eaf-95e4-287f58ff1c71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8630c7be7ea49d5b6d3ef6d46fd653e', 'request_id': 'req-ba8d3b30-6a4e-4eaf-95e4-287f58ff1c71'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:23.874769 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ba8d3b30-6a4e-4eaf-95e4-287f58ff1c71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8630c7be7ea49d5b6d3ef6d46fd653e', 'request_id': 'req-ba8d3b30-6a4e-4eaf-95e4-287f58ff1c71'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:23.881700 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ba8d3b30-6a4e-4eaf-95e4-287f58ff1c71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8630c7be7ea49d5b6d3ef6d46fd653e', 'request_id': 'req-ba8d3b30-6a4e-4eaf-95e4-287f58ff1c71'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:23.885898 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:16:23.895093 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ba8d3b30-6a4e-4eaf-95e4-287f58ff1c71 admin admin] The token cache is being invalidate because project a8630c7be7ea49d5b6d3ef6d46fd653e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:16:23.895455 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ba8d3b30-6a4e-4eaf-95e4-287f58ff1c71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:23.896398 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ba8d3b30-6a4e-4eaf-95e4-287f58ff1c71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:23.898121 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1132/2291] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:16:23 2022] DELETE /identity/v3/projects/a8630c7be7ea49d5b6d3ef6d46fd653e => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:16:23.902518 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f516e-7392-4025-bfd2-01b52dc20b73 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:23.904590 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-a13f516e-7392-4025-bfd2-01b52dc20b73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:23.917635 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f516e-7392-4025-bfd2-01b52dc20b73 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:23.973632 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb76500-72a2-4c2c-b89f-ca5b60d9a43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:23.974267 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb76500-72a2-4c2c-b89f-ca5b60d9a43d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:23.974542 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb76500-72a2-4c2c-b89f-ca5b60d9a43d admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:23.974818 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb76500-72a2-4c2c-b89f-ca5b60d9a43d admin admin] PATH_INFO: `/v3/users/e243686531424308bca5737102518edc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:23.975953 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb76500-72a2-4c2c-b89f-ca5b60d9a43d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e243686531424308bca5737102518edc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:23.977912 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb76500-72a2-4c2c-b89f-ca5b60d9a43d admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:24.025355 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-aeb76500-72a2-4c2c-b89f-ca5b60d9a43d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e243686531424308bca5737102518edc'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:24.026017 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-aeb76500-72a2-4c2c-b89f-ca5b60d9a43d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e243686531424308bca5737102518edc'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:24.038974 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-aeb76500-72a2-4c2c-b89f-ca5b60d9a43d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e243686531424308bca5737102518edc'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:24.047572 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-aeb76500-72a2-4c2c-b89f-ca5b60d9a43d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e243686531424308bca5737102518edc'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:24.048286 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-aeb76500-72a2-4c2c-b89f-ca5b60d9a43d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e243686531424308bca5737102518edc'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:24.060753 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1160/2292] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:16:23 2022] DELETE /identity/v3/users/e243686531424308bca5737102518edc => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:16:24.230060 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d3b30-6a4e-4eaf-95e4-287f58ff1c71 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:24.232020 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ba8d3b30-6a4e-4eaf-95e4-287f58ff1c71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:24.244422 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d3b30-6a4e-4eaf-95e4-287f58ff1c71 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:24.303053 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e556a-a8ac-4ca7-be90-0910edfb8dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:24.303386 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e556a-a8ac-4ca7-be90-0910edfb8dd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:24.303536 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e556a-a8ac-4ca7-be90-0910edfb8dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:24.303814 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e556a-a8ac-4ca7-be90-0910edfb8dd4 admin admin] PATH_INFO: `/v3/projects/cac144718edb4e0aaf75e7443eae2bf0` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:24.305016 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e556a-a8ac-4ca7-be90-0910edfb8dd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cac144718edb4e0aaf75e7443eae2bf0)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:24.306914 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e556a-a8ac-4ca7-be90-0910edfb8dd4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:24.357048 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-259e556a-a8ac-4ca7-be90-0910edfb8dd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cac144718edb4e0aaf75e7443eae2bf0', 'request_id': 'req-259e556a-a8ac-4ca7-be90-0910edfb8dd4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:24.364590 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-259e556a-a8ac-4ca7-be90-0910edfb8dd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cac144718edb4e0aaf75e7443eae2bf0', 'request_id': 'req-259e556a-a8ac-4ca7-be90-0910edfb8dd4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:24.371120 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-259e556a-a8ac-4ca7-be90-0910edfb8dd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cac144718edb4e0aaf75e7443eae2bf0', 'request_id': 'req-259e556a-a8ac-4ca7-be90-0910edfb8dd4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:24.382478 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-259e556a-a8ac-4ca7-be90-0910edfb8dd4 admin admin] The token cache is being invalidate because project cac144718edb4e0aaf75e7443eae2bf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:16:24.382775 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-259e556a-a8ac-4ca7-be90-0910edfb8dd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:24.383445 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-259e556a-a8ac-4ca7-be90-0910edfb8dd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:16:24.385421 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1133/2293] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:16:24 2022] DELETE /identity/v3/projects/cac144718edb4e0aaf75e7443eae2bf0 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:16:27.360531 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda3eca-d57f-4910-a03a-53629b085747 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:27.361020 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda3eca-d57f-4910-a03a-53629b085747 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:27.361142 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda3eca-d57f-4910-a03a-53629b085747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:27.381250 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-9dda3eca-d57f-4910-a03a-53629b085747 None None] MFA Rules not processed for user `2eb09df64dd74ddeb9fbbf44ae0a3592`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:16:27.382500 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-9dda3eca-d57f-4910-a03a-53629b085747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:27.394870 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:16:27.444948 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1161/2294] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:16:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:16:34.379248 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:DELETE=8,keystone:SELECT=53,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:16:35.361098 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e556a-a8ac-4ca7-be90-0910edfb8dd4 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:35.362561 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-259e556a-a8ac-4ca7-be90-0910edfb8dd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:35.437326 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e556a-a8ac-4ca7-be90-0910edfb8dd4 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:35.443143 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caef4ebf-9a5d-46b5-a4b8-2be2ccc94f74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:35.443809 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef4ebf-9a5d-46b5-a4b8-2be2ccc94f74 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:35.444156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef4ebf-9a5d-46b5-a4b8-2be2ccc94f74 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:35.444425 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef4ebf-9a5d-46b5-a4b8-2be2ccc94f74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:35.446220 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-caef4ebf-9a5d-46b5-a4b8-2be2ccc94f74 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:35.533020 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caef4ebf-9a5d-46b5-a4b8-2be2ccc94f74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:35.535791 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caef4ebf-9a5d-46b5-a4b8-2be2ccc94f74 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:35.551689 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1134/2295] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:16:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:16:37.437601 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=18 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:16:39.239259 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dda3eca-d57f-4910-a03a-53629b085747 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:16:39.253051 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dda3eca-d57f-4910-a03a-53629b085747 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:16:39.258502 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b10e094-97a2-49ba-b444-1634c5cb32d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:16:39.259153 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b10e094-97a2-49ba-b444-1634c5cb32d4 service nova] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:16:39.259439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b10e094-97a2-49ba-b444-1634c5cb32d4 service nova] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:16:39.259720 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b10e094-97a2-49ba-b444-1634c5cb32d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:16:39.261623 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-4b10e094-97a2-49ba-b444-1634c5cb32d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:16:39.272248 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b10e094-97a2-49ba-b444-1634c5cb32d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:16:39.273910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b10e094-97a2-49ba-b444-1634c5cb32d4 service nova] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:16:39.282991 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1162/2296] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:16:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:16:45.544270 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=56 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:16:49.279028 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=6 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:17:04.570214 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caef4ebf-9a5d-46b5-a4b8-2be2ccc94f74 service nova] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:04.571990 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-caef4ebf-9a5d-46b5-a4b8-2be2ccc94f74 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:04.663074 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caef4ebf-9a5d-46b5-a4b8-2be2ccc94f74 service nova] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:04.667489 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b120360-1602-466d-b29d-e6f3d2429513 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:04.668169 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b120360-1602-466d-b29d-e6f3d2429513 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:04.668458 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b120360-1602-466d-b29d-e6f3d2429513 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:04.668858 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b120360-1602-466d-b29d-e6f3d2429513 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:04.670651 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8b120360-1602-466d-b29d-e6f3d2429513 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:04.777073 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b120360-1602-466d-b29d-e6f3d2429513 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:04.778521 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b120360-1602-466d-b29d-e6f3d2429513 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:04.799883 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1135/2297] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:17:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:17:08.001658 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b10e094-97a2-49ba-b444-1634c5cb32d4 service nova] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:08.003102 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-4b10e094-97a2-49ba-b444-1634c5cb32d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:08.056433 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b10e094-97a2-49ba-b444-1634c5cb32d4 service nova] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:08.061033 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986c225b-5c56-4ff8-894c-43b208ab5ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:08.061674 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c225b-5c56-4ff8-894c-43b208ab5ad9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:08.061959 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c225b-5c56-4ff8-894c-43b208ab5ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:08.062240 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c225b-5c56-4ff8-894c-43b208ab5ad9 admin admin] PATH_INFO: `/v3/users/f379b005418a4e399c3b72cbbef6e00d` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:08.063414 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986c225b-5c56-4ff8-894c-43b208ab5ad9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f379b005418a4e399c3b72cbbef6e00d)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:08.065385 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986c225b-5c56-4ff8-894c-43b208ab5ad9 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:08.115496 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-986c225b-5c56-4ff8-894c-43b208ab5ad9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f379b005418a4e399c3b72cbbef6e00d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:08.116253 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-986c225b-5c56-4ff8-894c-43b208ab5ad9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f379b005418a4e399c3b72cbbef6e00d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:08.130778 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-986c225b-5c56-4ff8-894c-43b208ab5ad9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f379b005418a4e399c3b72cbbef6e00d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:08.138884 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-986c225b-5c56-4ff8-894c-43b208ab5ad9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f379b005418a4e399c3b72cbbef6e00d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:08.139657 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-986c225b-5c56-4ff8-894c-43b208ab5ad9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f379b005418a4e399c3b72cbbef6e00d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:08.151727 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1163/2298] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:17:08 2022] DELETE /identity/v3/users/f379b005418a4e399c3b72cbbef6e00d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:17:08.323068 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b120360-1602-466d-b29d-e6f3d2429513 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:08.339932 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b120360-1602-466d-b29d-e6f3d2429513 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:08.374083 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:17:08.401537 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703fb959-9c09-40cc-8dde-e58f41890a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:08.402181 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fb959-9c09-40cc-8dde-e58f41890a99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:08.402462 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fb959-9c09-40cc-8dde-e58f41890a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:08.402734 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fb959-9c09-40cc-8dde-e58f41890a99 admin admin] PATH_INFO: `/v3/projects/670dbffe4aed43c6afc3899ff59b8040` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:08.403878 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703fb959-9c09-40cc-8dde-e58f41890a99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=670dbffe4aed43c6afc3899ff59b8040)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:08.405844 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703fb959-9c09-40cc-8dde-e58f41890a99 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:08.453475 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-703fb959-9c09-40cc-8dde-e58f41890a99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '670dbffe4aed43c6afc3899ff59b8040', 'request_id': 'req-703fb959-9c09-40cc-8dde-e58f41890a99'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:08.460546 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-703fb959-9c09-40cc-8dde-e58f41890a99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '670dbffe4aed43c6afc3899ff59b8040', 'request_id': 'req-703fb959-9c09-40cc-8dde-e58f41890a99'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:08.466770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-703fb959-9c09-40cc-8dde-e58f41890a99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '670dbffe4aed43c6afc3899ff59b8040', 'request_id': 'req-703fb959-9c09-40cc-8dde-e58f41890a99'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:08.477079 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-703fb959-9c09-40cc-8dde-e58f41890a99 admin admin] The token cache is being invalidate because project 670dbffe4aed43c6afc3899ff59b8040 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:17:08.477884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-703fb959-9c09-40cc-8dde-e58f41890a99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:08.478508 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-703fb959-9c09-40cc-8dde-e58f41890a99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:08.480004 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1136/2299] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:17:08 2022] DELETE /identity/v3/projects/670dbffe4aed43c6afc3899ff59b8040 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:17:08.484960 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba05ff-8623-44aa-8d17-eacbc16c7ee7 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:08.485265 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba05ff-8623-44aa-8d17-eacbc16c7ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:08.485551 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba05ff-8623-44aa-8d17-eacbc16c7ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:08.542087 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-4aba05ff-8623-44aa-8d17-eacbc16c7ee7 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:17:08.543298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-4aba05ff-8623-44aa-8d17-eacbc16c7ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:08.604940 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1164/2300] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:17:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:17:08.613068 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7c518-c3ed-4b8f-aca1-36d959e7d0ee None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:08.613221 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7c518-c3ed-4b8f-aca1-36d959e7d0ee None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:08.613504 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7c518-c3ed-4b8f-aca1-36d959e7d0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:08.633745 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-dea7c518-c3ed-4b8f-aca1-36d959e7d0ee None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:17:08.635080 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-dea7c518-c3ed-4b8f-aca1-36d959e7d0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:08.643389 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1137/2301] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:17:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:17:08.647604 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba05ff-8623-44aa-8d17-eacbc16c7ee7 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:08.657278 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba05ff-8623-44aa-8d17-eacbc16c7ee7 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:08.661970 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6003c-ce2b-44d1-8118-c23db0516fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:08.662668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6003c-ce2b-44d1-8118-c23db0516fae admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:08.663023 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6003c-ce2b-44d1-8118-c23db0516fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:08.663361 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6003c-ce2b-44d1-8118-c23db0516fae admin admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:08.664128 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed6003c-ce2b-44d1-8118-c23db0516fae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:17:08.664530 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed6003c-ce2b-44d1-8118-c23db0516fae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:08.666645 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed6003c-ce2b-44d1-8118-c23db0516fae admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:08.669184 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:17:08.685028 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1165/2302] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:17:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:17:08.685735 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7c518-c3ed-4b8f-aca1-36d959e7d0ee None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:08.695868 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7c518-c3ed-4b8f-aca1-36d959e7d0ee None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:08.700882 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c95f38-159f-46c2-855c-25fb3cfeff76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:08.701523 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c95f38-159f-46c2-855c-25fb3cfeff76 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:08.701831 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c95f38-159f-46c2-855c-25fb3cfeff76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:08.702111 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c95f38-159f-46c2-855c-25fb3cfeff76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:08.702779 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c95f38-159f-46c2-855c-25fb3cfeff76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:08.704747 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c95f38-159f-46c2-855c-25fb3cfeff76 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:08.731667 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1138/2303] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:17:08 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:17:08.736017 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6003c-ce2b-44d1-8118-c23db0516fae admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:08.746656 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6003c-ce2b-44d1-8118-c23db0516fae admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:08.801532 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3613aeb-67fc-4a59-b8da-edfb8f38dfda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:08.802158 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3613aeb-67fc-4a59-b8da-edfb8f38dfda admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:08.802447 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3613aeb-67fc-4a59-b8da-edfb8f38dfda admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:08.802726 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3613aeb-67fc-4a59-b8da-edfb8f38dfda admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:08.803400 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3613aeb-67fc-4a59-b8da-edfb8f38dfda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:08.805358 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3613aeb-67fc-4a59-b8da-edfb8f38dfda admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:08.828847 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1166/2304] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:17:08 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:17:08.832932 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c95f38-159f-46c2-855c-25fb3cfeff76 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:08.843785 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c95f38-159f-46c2-855c-25fb3cfeff76 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:08.848386 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d0c622-2a70-4bff-b9b2-bad4514d4ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:08.849022 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d0c622-2a70-4bff-b9b2-bad4514d4ac4 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:08.849306 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d0c622-2a70-4bff-b9b2-bad4514d4ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:08.849600 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d0c622-2a70-4bff-b9b2-bad4514d4ac4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:08.850280 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d0c622-2a70-4bff-b9b2-bad4514d4ac4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:08.851859 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d0c622-2a70-4bff-b9b2-bad4514d4ac4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:08.862469 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1139/2305] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:17:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:17:08.866376 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3613aeb-67fc-4a59-b8da-edfb8f38dfda admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:08.876253 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3613aeb-67fc-4a59-b8da-edfb8f38dfda admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:08.880790 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba25f5d-93ea-488b-9bf7-f67e57374702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:08.881390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba25f5d-93ea-488b-9bf7-f67e57374702 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:08.881672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba25f5d-93ea-488b-9bf7-f67e57374702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:08.881946 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba25f5d-93ea-488b-9bf7-f67e57374702 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:08.882635 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba25f5d-93ea-488b-9bf7-f67e57374702 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:08.884259 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba25f5d-93ea-488b-9bf7-f67e57374702 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:08.894942 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1167/2306] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:17:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:17:08.898784 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d0c622-2a70-4bff-b9b2-bad4514d4ac4 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:08.908983 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d0c622-2a70-4bff-b9b2-bad4514d4ac4 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:08.913576 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3fa0c-7a3a-4157-b463-07e0d72ac1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:08.914292 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3fa0c-7a3a-4157-b463-07e0d72ac1d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:08.914537 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3fa0c-7a3a-4157-b463-07e0d72ac1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:08.915388 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3fa0c-7a3a-4157-b463-07e0d72ac1d7 admin admin] PATH_INFO: `/v3/projects/b52bbd619aab41a2a5db484a67f4ee66/users/28314ed2f77c4d6f8edfd35941d4d554/roles/926b9f36c24847c582c84af904250abc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:08.929595 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c3fa0c-7a3a-4157-b463-07e0d72ac1d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b52bbd619aab41a2a5db484a67f4ee66, user_id=28314ed2f77c4d6f8edfd35941d4d554, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:08.932219 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c3fa0c-7a3a-4157-b463-07e0d72ac1d7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:08.944942 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1140/2307] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:17:08 2022] PUT /identity/v3/projects/b52bbd619aab41a2a5db484a67f4ee66/users/28314ed2f77c4d6f8edfd35941d4d554/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:17:08.956508 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1428c-d99e-4314-a6be-e0b1de8b4c31 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:08.956893 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1428c-d99e-4314-a6be-e0b1de8b4c31 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:08.957244 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1428c-d99e-4314-a6be-e0b1de8b4c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:08.989159 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-41b1428c-d99e-4314-a6be-e0b1de8b4c31 None None] MFA Rules not processed for user `28314ed2f77c4d6f8edfd35941d4d554`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:17:08.990380 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-41b1428c-d99e-4314-a6be-e0b1de8b4c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:09.053684 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1168/2308] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:17:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:17:09.065302 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3fa0c-7a3a-4157-b463-07e0d72ac1d7 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:09.066970 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f7c3fa0c-7a3a-4157-b463-07e0d72ac1d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:09.127597 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:17:09.168669 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3fa0c-7a3a-4157-b463-07e0d72ac1d7 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:09.174169 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc432f2d-3d8f-41a3-a9a2-4004afbd47b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:09.174783 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc432f2d-3d8f-41a3-a9a2-4004afbd47b7 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:09.175069 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc432f2d-3d8f-41a3-a9a2-4004afbd47b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:09.175348 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc432f2d-3d8f-41a3-a9a2-4004afbd47b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:09.183445 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc432f2d-3d8f-41a3-a9a2-4004afbd47b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:09.185056 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc432f2d-3d8f-41a3-a9a2-4004afbd47b7 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:09.193723 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1141/2309] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:17:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:17:18.547703 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1428c-d99e-4314-a6be-e0b1de8b4c31 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:18.609184 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1428c-d99e-4314-a6be-e0b1de8b4c31 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:18.613643 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00e82aa-05ad-4854-819d-f72fcb342896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:18.614274 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e82aa-05ad-4854-819d-f72fcb342896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:18.614558 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e82aa-05ad-4854-819d-f72fcb342896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:18.614835 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e82aa-05ad-4854-819d-f72fcb342896 admin admin] PATH_INFO: `/v3/users/28314ed2f77c4d6f8edfd35941d4d554` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:18.616019 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00e82aa-05ad-4854-819d-f72fcb342896 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28314ed2f77c4d6f8edfd35941d4d554)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:18.618487 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00e82aa-05ad-4854-819d-f72fcb342896 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:18.622980 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:17:18.666298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-e00e82aa-05ad-4854-819d-f72fcb342896 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28314ed2f77c4d6f8edfd35941d4d554'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:18.666982 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-e00e82aa-05ad-4854-819d-f72fcb342896 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28314ed2f77c4d6f8edfd35941d4d554'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:18.678608 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-e00e82aa-05ad-4854-819d-f72fcb342896 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28314ed2f77c4d6f8edfd35941d4d554'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:18.686039 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-e00e82aa-05ad-4854-819d-f72fcb342896 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28314ed2f77c4d6f8edfd35941d4d554'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:18.686664 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-e00e82aa-05ad-4854-819d-f72fcb342896 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28314ed2f77c4d6f8edfd35941d4d554'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:18.699967 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1169/2310] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:17:18 2022] DELETE /identity/v3/users/28314ed2f77c4d6f8edfd35941d4d554 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:17:18.704415 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000fd214-756c-4486-9e61-c6ceecea2eec None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:18.704911 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000fd214-756c-4486-9e61-c6ceecea2eec None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:18.705035 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000fd214-756c-4486-9e61-c6ceecea2eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:18.723811 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-000fd214-756c-4486-9e61-c6ceecea2eec None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:17:18.725043 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-000fd214-756c-4486-9e61-c6ceecea2eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:18.783253 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1142/2311] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:17:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:17:18.793026 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00e82aa-05ad-4854-819d-f72fcb342896 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:18.794523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-e00e82aa-05ad-4854-819d-f72fcb342896 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:18.879293 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00e82aa-05ad-4854-819d-f72fcb342896 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:18.883514 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4165c07-f076-4503-ab65-bab13ebe19ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:18.884128 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4165c07-f076-4503-ab65-bab13ebe19ec service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:18.884409 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4165c07-f076-4503-ab65-bab13ebe19ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:18.884715 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4165c07-f076-4503-ab65-bab13ebe19ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:18.893697 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4165c07-f076-4503-ab65-bab13ebe19ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:18.895119 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4165c07-f076-4503-ab65-bab13ebe19ec service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:18.904442 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1170/2312] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:17:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:17:19.174129 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000fd214-756c-4486-9e61-c6ceecea2eec None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:19.175546 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-000fd214-756c-4486-9e61-c6ceecea2eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:19.188081 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000fd214-756c-4486-9e61-c6ceecea2eec None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:19.192682 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21ce6b-0248-4249-b319-c0691d58bfa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:19.193331 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21ce6b-0248-4249-b319-c0691d58bfa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:19.193616 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21ce6b-0248-4249-b319-c0691d58bfa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:19.193911 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21ce6b-0248-4249-b319-c0691d58bfa3 admin admin] PATH_INFO: `/v3/projects/b52bbd619aab41a2a5db484a67f4ee66` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:19.195059 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21ce6b-0248-4249-b319-c0691d58bfa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b52bbd619aab41a2a5db484a67f4ee66)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:19.197253 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21ce6b-0248-4249-b319-c0691d58bfa3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:19.244141 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ae21ce6b-0248-4249-b319-c0691d58bfa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b52bbd619aab41a2a5db484a67f4ee66', 'request_id': 'req-ae21ce6b-0248-4249-b319-c0691d58bfa3'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:19.251177 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ae21ce6b-0248-4249-b319-c0691d58bfa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b52bbd619aab41a2a5db484a67f4ee66', 'request_id': 'req-ae21ce6b-0248-4249-b319-c0691d58bfa3'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:19.257327 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ae21ce6b-0248-4249-b319-c0691d58bfa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b52bbd619aab41a2a5db484a67f4ee66', 'request_id': 'req-ae21ce6b-0248-4249-b319-c0691d58bfa3'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:19.267466 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ae21ce6b-0248-4249-b319-c0691d58bfa3 admin admin] The token cache is being invalidate because project b52bbd619aab41a2a5db484a67f4ee66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:17:19.267757 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ae21ce6b-0248-4249-b319-c0691d58bfa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:19.268392 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ae21ce6b-0248-4249-b319-c0691d58bfa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:17:19.269890 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1143/2313] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:17:19 2022] DELETE /identity/v3/projects/b52bbd619aab41a2a5db484a67f4ee66 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:17:19.274916 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aeb3c0-7a0c-4cad-ba2e-5ec9789a4d76 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:19.275206 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aeb3c0-7a0c-4cad-ba2e-5ec9789a4d76 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:19.275512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aeb3c0-7a0c-4cad-ba2e-5ec9789a4d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:19.294973 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-c8aeb3c0-7a0c-4cad-ba2e-5ec9789a4d76 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:17:19.296172 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-c8aeb3c0-7a0c-4cad-ba2e-5ec9789a4d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:19.354592 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1171/2314] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:17:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:17:19.362382 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c2b3c-f058-4d45-9694-337ec86e7586 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:19.362673 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c2b3c-f058-4d45-9694-337ec86e7586 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:19.362952 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c2b3c-f058-4d45-9694-337ec86e7586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:19.382402 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5d1c2b3c-f058-4d45-9694-337ec86e7586 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:17:19.383589 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5d1c2b3c-f058-4d45-9694-337ec86e7586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:19.391297 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1144/2315] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:17:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:17:19.395420 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aeb3c0-7a0c-4cad-ba2e-5ec9789a4d76 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:19.405784 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aeb3c0-7a0c-4cad-ba2e-5ec9789a4d76 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:19.410339 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b94a0f6-e5f5-4bd0-a0d5-a5afbb396560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:19.410948 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b94a0f6-e5f5-4bd0-a0d5-a5afbb396560 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:19.411234 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b94a0f6-e5f5-4bd0-a0d5-a5afbb396560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:19.411513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b94a0f6-e5f5-4bd0-a0d5-a5afbb396560 admin admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:19.412208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b94a0f6-e5f5-4bd0-a0d5-a5afbb396560 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:17:19.412909 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b94a0f6-e5f5-4bd0-a0d5-a5afbb396560 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:19.414801 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b94a0f6-e5f5-4bd0-a0d5-a5afbb396560 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:19.416271 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:17:19.428245 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1172/2316] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:17:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:17:19.432264 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c2b3c-f058-4d45-9694-337ec86e7586 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:19.441654 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c2b3c-f058-4d45-9694-337ec86e7586 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:19.448451 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ff7c30-2714-4dd4-adbb-ebeaf536c7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:19.449181 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ff7c30-2714-4dd4-adbb-ebeaf536c7a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:19.449544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ff7c30-2714-4dd4-adbb-ebeaf536c7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:19.450365 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ff7c30-2714-4dd4-adbb-ebeaf536c7a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:19.451121 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ff7c30-2714-4dd4-adbb-ebeaf536c7a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:19.453163 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ff7c30-2714-4dd4-adbb-ebeaf536c7a9 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:19.481506 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1145/2317] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:17:19 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:17:19.485660 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b94a0f6-e5f5-4bd0-a0d5-a5afbb396560 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:19.495749 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b94a0f6-e5f5-4bd0-a0d5-a5afbb396560 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:19.553642 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bdb2c4-2416-40f9-8557-1365b2e2698c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:19.554381 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bdb2c4-2416-40f9-8557-1365b2e2698c admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:19.554566 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bdb2c4-2416-40f9-8557-1365b2e2698c admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:19.554849 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bdb2c4-2416-40f9-8557-1365b2e2698c admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:19.555513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bdb2c4-2416-40f9-8557-1365b2e2698c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:19.557470 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bdb2c4-2416-40f9-8557-1365b2e2698c admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:19.578736 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1173/2318] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:17:19 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:17:19.582657 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ff7c30-2714-4dd4-adbb-ebeaf536c7a9 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:19.593344 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ff7c30-2714-4dd4-adbb-ebeaf536c7a9 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:19.598127 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8686e2-59d8-488c-8ceb-ea0775004689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:19.598750 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8686e2-59d8-488c-8ceb-ea0775004689 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:19.599062 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8686e2-59d8-488c-8ceb-ea0775004689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:19.599343 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8686e2-59d8-488c-8ceb-ea0775004689 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:19.600071 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8686e2-59d8-488c-8ceb-ea0775004689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:19.602390 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8686e2-59d8-488c-8ceb-ea0775004689 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:19.612259 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1146/2319] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:17:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:17:19.616055 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bdb2c4-2416-40f9-8557-1365b2e2698c admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:19.625885 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bdb2c4-2416-40f9-8557-1365b2e2698c admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:19.630451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d2c6f6-1e8c-4546-a664-a1798ef59b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:19.631060 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2c6f6-1e8c-4546-a664-a1798ef59b58 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:19.631347 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2c6f6-1e8c-4546-a664-a1798ef59b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:19.631627 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2c6f6-1e8c-4546-a664-a1798ef59b58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:19.632345 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d2c6f6-1e8c-4546-a664-a1798ef59b58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:19.634050 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d2c6f6-1e8c-4546-a664-a1798ef59b58 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:19.644995 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1174/2320] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:17:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:17:19.649392 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8686e2-59d8-488c-8ceb-ea0775004689 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:19.654488 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:17:19.660849 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8686e2-59d8-488c-8ceb-ea0775004689 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:19.667911 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99419cb-4141-4abb-98d1-b9ba1ff3c946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:19.668692 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99419cb-4141-4abb-98d1-b9ba1ff3c946 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:19.669047 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99419cb-4141-4abb-98d1-b9ba1ff3c946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:19.669403 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99419cb-4141-4abb-98d1-b9ba1ff3c946 admin admin] PATH_INFO: `/v3/projects/00a6d025e9ff4eb69b472330c4f3846c/users/e171d38b836d42b187659780810ec71b/roles/926b9f36c24847c582c84af904250abc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:19.684267 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99419cb-4141-4abb-98d1-b9ba1ff3c946 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00a6d025e9ff4eb69b472330c4f3846c, user_id=e171d38b836d42b187659780810ec71b, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:19.687104 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99419cb-4141-4abb-98d1-b9ba1ff3c946 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:19.701549 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1147/2321] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:17:19 2022] PUT /identity/v3/projects/00a6d025e9ff4eb69b472330c4f3846c/users/e171d38b836d42b187659780810ec71b/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:17:19.712312 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577ce1c-c0d6-4e7d-b428-57fe65befbd9 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:19.712639 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577ce1c-c0d6-4e7d-b428-57fe65befbd9 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:19.712920 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577ce1c-c0d6-4e7d-b428-57fe65befbd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:19.741949 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-4577ce1c-c0d6-4e7d-b428-57fe65befbd9 None None] MFA Rules not processed for user `e171d38b836d42b187659780810ec71b`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:17:19.743167 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-4577ce1c-c0d6-4e7d-b428-57fe65befbd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:19.812049 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1175/2322] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:17:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:17:19.816707 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99419cb-4141-4abb-98d1-b9ba1ff3c946 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:19.885051 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99419cb-4141-4abb-98d1-b9ba1ff3c946 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:19.889727 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6199a0af-e73a-4d57-8551-becd9f085012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:19.890379 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199a0af-e73a-4d57-8551-becd9f085012 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:19.890692 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199a0af-e73a-4d57-8551-becd9f085012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:19.890968 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199a0af-e73a-4d57-8551-becd9f085012 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:19.891689 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6199a0af-e73a-4d57-8551-becd9f085012 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:19.893690 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6199a0af-e73a-4d57-8551-becd9f085012 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:19.924546 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1148/2323] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:17:19 2022] POST /identity/v3/projects => generated 371 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:17:19.925801 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4577ce1c-c0d6-4e7d-b428-57fe65befbd9 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:19.937653 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4577ce1c-c0d6-4e7d-b428-57fe65befbd9 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:19.993748 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b908e-5660-4b44-865c-9ff17cb59213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:19.994391 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b908e-5660-4b44-865c-9ff17cb59213 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:19.994675 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b908e-5660-4b44-865c-9ff17cb59213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:19.994953 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b908e-5660-4b44-865c-9ff17cb59213 admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:19.995622 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51b908e-5660-4b44-865c-9ff17cb59213 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:19.997575 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51b908e-5660-4b44-865c-9ff17cb59213 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:20.006293 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:17:20.019957 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1176/2324] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:17:19 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:17:20.024093 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6199a0af-e73a-4d57-8551-becd9f085012 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:20.034494 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6199a0af-e73a-4d57-8551-becd9f085012 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:20.039605 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c3e86-c2d8-4484-813b-af1e31771613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:20.040216 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c3e86-c2d8-4484-813b-af1e31771613 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:20.040539 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c3e86-c2d8-4484-813b-af1e31771613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:20.040822 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c3e86-c2d8-4484-813b-af1e31771613 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:20.041522 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c3e86-c2d8-4484-813b-af1e31771613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:20.043121 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c3e86-c2d8-4484-813b-af1e31771613 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:20.053527 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1149/2325] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:17:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:17:20.058197 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b908e-5660-4b44-865c-9ff17cb59213 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:20.068197 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b908e-5660-4b44-865c-9ff17cb59213 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:20.073139 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28fc6d6-ee5e-4f6e-858b-5312a2a5e558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:20.073803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28fc6d6-ee5e-4f6e-858b-5312a2a5e558 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:20.074091 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28fc6d6-ee5e-4f6e-858b-5312a2a5e558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:20.074379 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28fc6d6-ee5e-4f6e-858b-5312a2a5e558 admin admin] PATH_INFO: `/v3/projects/56992dcba15b4320a68bbe51d1467ec0/users/c8314c5f101f402bb71245e39ed6eafe/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:20.090400 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28fc6d6-ee5e-4f6e-858b-5312a2a5e558 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56992dcba15b4320a68bbe51d1467ec0, user_id=c8314c5f101f402bb71245e39ed6eafe, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:20.093097 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28fc6d6-ee5e-4f6e-858b-5312a2a5e558 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:20.106393 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1177/2326] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:17:20 2022] PUT /identity/v3/projects/56992dcba15b4320a68bbe51d1467ec0/users/c8314c5f101f402bb71245e39ed6eafe/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:17:20.110212 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c3e86-c2d8-4484-813b-af1e31771613 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:20.120181 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c3e86-c2d8-4484-813b-af1e31771613 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:20.175582 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaa4a51-55d1-4f16-86e1-ea6a89ef39fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:20.176201 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa4a51-55d1-4f16-86e1-ea6a89ef39fc admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:20.176511 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa4a51-55d1-4f16-86e1-ea6a89ef39fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:20.176799 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa4a51-55d1-4f16-86e1-ea6a89ef39fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:20.177494 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaa4a51-55d1-4f16-86e1-ea6a89ef39fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:20.179074 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaa4a51-55d1-4f16-86e1-ea6a89ef39fc admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:20.188700 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1150/2327] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:17:20 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:17:20.192780 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28fc6d6-ee5e-4f6e-858b-5312a2a5e558 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:20.203241 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28fc6d6-ee5e-4f6e-858b-5312a2a5e558 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:20.207805 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a3464-2317-486a-8841-4479c5c58835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:20.208465 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a3464-2317-486a-8841-4479c5c58835 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:20.208774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a3464-2317-486a-8841-4479c5c58835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:20.209056 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a3464-2317-486a-8841-4479c5c58835 admin admin] PATH_INFO: `/v3/projects/56992dcba15b4320a68bbe51d1467ec0/users/c8314c5f101f402bb71245e39ed6eafe/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:20.211457 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a3464-2317-486a-8841-4479c5c58835 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56992dcba15b4320a68bbe51d1467ec0, user_id=c8314c5f101f402bb71245e39ed6eafe, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:20.214113 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a3464-2317-486a-8841-4479c5c58835 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:20.224070 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1178/2328] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:17:20 2022] PUT /identity/v3/projects/56992dcba15b4320a68bbe51d1467ec0/users/c8314c5f101f402bb71245e39ed6eafe/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:17:20.236269 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69645e-360e-4d3f-851c-eee7fdabd9a3 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:20.236597 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69645e-360e-4d3f-851c-eee7fdabd9a3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:20.236883 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69645e-360e-4d3f-851c-eee7fdabd9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:20.263929 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-6d69645e-360e-4d3f-851c-eee7fdabd9a3 None None] MFA Rules not processed for user `c8314c5f101f402bb71245e39ed6eafe`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:17:20.265180 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6d69645e-360e-4d3f-851c-eee7fdabd9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:20.302337 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:17:20.333395 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1151/2329] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:17:20 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:17:20.345021 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a3464-2317-486a-8841-4479c5c58835 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:20.346755 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-115a3464-2317-486a-8841-4479c5c58835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:17:20.424442 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a3464-2317-486a-8841-4479c5c58835 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:20.428225 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3011b9-c5f6-49c1-a164-5253c4930bb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:20.428902 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3011b9-c5f6-49c1-a164-5253c4930bb8 service cinder] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:20.429190 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3011b9-c5f6-49c1-a164-5253c4930bb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:20.430074 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3011b9-c5f6-49c1-a164-5253c4930bb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:20.439085 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3011b9-c5f6-49c1-a164-5253c4930bb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:20.440450 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3011b9-c5f6-49c1-a164-5253c4930bb8 service cinder] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:20.450984 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1179/2330] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:17:20 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:17:20.523301 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d69645e-360e-4d3f-851c-eee7fdabd9a3 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:17:20.533207 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d69645e-360e-4d3f-851c-eee7fdabd9a3 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:17:20.536814 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f6b7bd-3f9d-4e22-8e9f-ff6b4874bc02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:17:20.537429 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6b7bd-3f9d-4e22-8e9f-ff6b4874bc02 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:17:20.537715 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6b7bd-3f9d-4e22-8e9f-ff6b4874bc02 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:17:20.537992 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6b7bd-3f9d-4e22-8e9f-ff6b4874bc02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:17:20.594081 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f6b7bd-3f9d-4e22-8e9f-ff6b4874bc02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:17:20.595498 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f6b7bd-3f9d-4e22-8e9f-ff6b4874bc02 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:17:20.604750 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1152/2331] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:17:20 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:17:30.445769 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:INSERT=4,keystone:SELECT=52 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:17:30.600177 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=40 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:03.206598 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3011b9-c5f6-49c1-a164-5253c4930bb8 service cinder] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:03.208168 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-cf3011b9-c5f6-49c1-a164-5253c4930bb8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:03.304374 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3011b9-c5f6-49c1-a164-5253c4930bb8 service cinder] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:03.310079 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4bdfc-4afb-475d-89a3-6932bccef8db service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e294fe796a1a4ac89041a532856ad627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:03.310699 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4bdfc-4afb-475d-89a3-6932bccef8db service placement] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:03.311047 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4bdfc-4afb-475d-89a3-6932bccef8db service placement] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:03.311329 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4bdfc-4afb-475d-89a3-6932bccef8db service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:03.313305 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-aad4bdfc-4afb-475d-89a3-6932bccef8db service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:03.327893 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad4bdfc-4afb-475d-89a3-6932bccef8db service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:03.329570 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad4bdfc-4afb-475d-89a3-6932bccef8db service placement] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:03.339684 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1180/2332] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:18:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:18:06.166476 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f6b7bd-3f9d-4e22-8e9f-ff6b4874bc02 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:06.229530 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f6b7bd-3f9d-4e22-8e9f-ff6b4874bc02 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:06.234564 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e789361-093c-41e7-84da-932fec99a6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:06.235198 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e789361-093c-41e7-84da-932fec99a6a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:06.235484 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e789361-093c-41e7-84da-932fec99a6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:06.235765 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e789361-093c-41e7-84da-932fec99a6a2 admin admin] PATH_INFO: `/v3/users/e171d38b836d42b187659780810ec71b` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:06.236948 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e789361-093c-41e7-84da-932fec99a6a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e171d38b836d42b187659780810ec71b)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:06.238855 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e789361-093c-41e7-84da-932fec99a6a2 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:06.283937 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0e789361-093c-41e7-84da-932fec99a6a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e171d38b836d42b187659780810ec71b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:06.290735 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0e789361-093c-41e7-84da-932fec99a6a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e171d38b836d42b187659780810ec71b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:06.291402 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0e789361-093c-41e7-84da-932fec99a6a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e171d38b836d42b187659780810ec71b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:06.298256 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0e789361-093c-41e7-84da-932fec99a6a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e171d38b836d42b187659780810ec71b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:06.299072 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0e789361-093c-41e7-84da-932fec99a6a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e171d38b836d42b187659780810ec71b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:06.316751 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1153/2333] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:18:06 2022] DELETE /identity/v3/users/e171d38b836d42b187659780810ec71b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:06.321550 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e3ab9-89e3-40d7-91f4-6649f9d19655 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:06.322017 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e3ab9-89e3-40d7-91f4-6649f9d19655 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:06.322116 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e3ab9-89e3-40d7-91f4-6649f9d19655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:06.344561 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-977e3ab9-89e3-40d7-91f4-6649f9d19655 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:06.345775 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-977e3ab9-89e3-40d7-91f4-6649f9d19655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:06.403867 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1181/2334] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:18:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:06.413573 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e789361-093c-41e7-84da-932fec99a6a2 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:06.414987 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0e789361-093c-41e7-84da-932fec99a6a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:06.472775 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e789361-093c-41e7-84da-932fec99a6a2 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:06.476309 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd462be9-a919-412c-ad27-6e7230aaa183 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:06.476965 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd462be9-a919-412c-ad27-6e7230aaa183 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:06.477247 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd462be9-a919-412c-ad27-6e7230aaa183 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:06.477522 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd462be9-a919-412c-ad27-6e7230aaa183 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:06.486854 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd462be9-a919-412c-ad27-6e7230aaa183 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:06.488268 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd462be9-a919-412c-ad27-6e7230aaa183 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:06.497531 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1154/2335] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:18:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:18:06.769132 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e3ab9-89e3-40d7-91f4-6649f9d19655 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:06.771811 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-977e3ab9-89e3-40d7-91f4-6649f9d19655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:06.790220 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e3ab9-89e3-40d7-91f4-6649f9d19655 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:06.795339 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e6d4cf-bc79-4757-8d0a-50feb5aa4ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:06.796092 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6d4cf-bc79-4757-8d0a-50feb5aa4ed5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:06.796510 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6d4cf-bc79-4757-8d0a-50feb5aa4ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:06.796915 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6d4cf-bc79-4757-8d0a-50feb5aa4ed5 admin admin] PATH_INFO: `/v3/projects/00a6d025e9ff4eb69b472330c4f3846c` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:06.798359 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e6d4cf-bc79-4757-8d0a-50feb5aa4ed5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00a6d025e9ff4eb69b472330c4f3846c)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:06.800578 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e6d4cf-bc79-4757-8d0a-50feb5aa4ed5 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:06.849794 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-f7e6d4cf-bc79-4757-8d0a-50feb5aa4ed5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00a6d025e9ff4eb69b472330c4f3846c', 'request_id': 'req-f7e6d4cf-bc79-4757-8d0a-50feb5aa4ed5'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:06.855724 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-f7e6d4cf-bc79-4757-8d0a-50feb5aa4ed5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00a6d025e9ff4eb69b472330c4f3846c', 'request_id': 'req-f7e6d4cf-bc79-4757-8d0a-50feb5aa4ed5'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:06.861451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-f7e6d4cf-bc79-4757-8d0a-50feb5aa4ed5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00a6d025e9ff4eb69b472330c4f3846c', 'request_id': 'req-f7e6d4cf-bc79-4757-8d0a-50feb5aa4ed5'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:06.863943 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:06.875779 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-f7e6d4cf-bc79-4757-8d0a-50feb5aa4ed5 admin admin] The token cache is being invalidate because project 00a6d025e9ff4eb69b472330c4f3846c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:18:06.876063 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-f7e6d4cf-bc79-4757-8d0a-50feb5aa4ed5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:06.876721 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-f7e6d4cf-bc79-4757-8d0a-50feb5aa4ed5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:06.878237 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1182/2336] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:18:06 2022] DELETE /identity/v3/projects/00a6d025e9ff4eb69b472330c4f3846c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:06.882239 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd462be9-a919-412c-ad27-6e7230aaa183 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:06.892173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd462be9-a919-412c-ad27-6e7230aaa183 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:06.910170 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:06.950670 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd6472-b25f-4375-9e23-164056cfc012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:06.951391 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd6472-b25f-4375-9e23-164056cfc012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:06.951683 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd6472-b25f-4375-9e23-164056cfc012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:06.951965 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd6472-b25f-4375-9e23-164056cfc012 admin admin] PATH_INFO: `/v3/users/c8314c5f101f402bb71245e39ed6eafe` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:06.953207 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fd6472-b25f-4375-9e23-164056cfc012 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8314c5f101f402bb71245e39ed6eafe)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:06.955130 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fd6472-b25f-4375-9e23-164056cfc012 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:07.001414 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c3fd6472-b25f-4375-9e23-164056cfc012 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8314c5f101f402bb71245e39ed6eafe'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:07.008301 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c3fd6472-b25f-4375-9e23-164056cfc012 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8314c5f101f402bb71245e39ed6eafe'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:07.009003 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c3fd6472-b25f-4375-9e23-164056cfc012 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8314c5f101f402bb71245e39ed6eafe'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:07.016912 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c3fd6472-b25f-4375-9e23-164056cfc012 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8314c5f101f402bb71245e39ed6eafe'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:07.017529 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c3fd6472-b25f-4375-9e23-164056cfc012 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8314c5f101f402bb71245e39ed6eafe'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:07.035494 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1155/2337] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:18:06 2022] DELETE /identity/v3/users/c8314c5f101f402bb71245e39ed6eafe => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:07.302531 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e6d4cf-bc79-4757-8d0a-50feb5aa4ed5 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:07.303988 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-f7e6d4cf-bc79-4757-8d0a-50feb5aa4ed5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:07.318257 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e6d4cf-bc79-4757-8d0a-50feb5aa4ed5 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:07.374509 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce9e1d-e95e-423c-81f8-cb7a02f68093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:07.375154 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce9e1d-e95e-423c-81f8-cb7a02f68093 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:07.375415 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce9e1d-e95e-423c-81f8-cb7a02f68093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:07.375689 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce9e1d-e95e-423c-81f8-cb7a02f68093 admin admin] PATH_INFO: `/v3/projects/56992dcba15b4320a68bbe51d1467ec0` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:07.376880 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ce9e1d-e95e-423c-81f8-cb7a02f68093 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56992dcba15b4320a68bbe51d1467ec0)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:07.378827 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ce9e1d-e95e-423c-81f8-cb7a02f68093 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:07.424190 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-84ce9e1d-e95e-423c-81f8-cb7a02f68093 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56992dcba15b4320a68bbe51d1467ec0', 'request_id': 'req-84ce9e1d-e95e-423c-81f8-cb7a02f68093'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:07.430334 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-84ce9e1d-e95e-423c-81f8-cb7a02f68093 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56992dcba15b4320a68bbe51d1467ec0', 'request_id': 'req-84ce9e1d-e95e-423c-81f8-cb7a02f68093'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:07.435862 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-84ce9e1d-e95e-423c-81f8-cb7a02f68093 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56992dcba15b4320a68bbe51d1467ec0', 'request_id': 'req-84ce9e1d-e95e-423c-81f8-cb7a02f68093'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:07.447992 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-84ce9e1d-e95e-423c-81f8-cb7a02f68093 admin admin] The token cache is being invalidate because project 56992dcba15b4320a68bbe51d1467ec0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:18:07.448284 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-84ce9e1d-e95e-423c-81f8-cb7a02f68093 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:07.448932 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-84ce9e1d-e95e-423c-81f8-cb7a02f68093 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:07.450934 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1183/2338] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:18:07 2022] DELETE /identity/v3/projects/56992dcba15b4320a68bbe51d1467ec0 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:07.456929 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4133f5-958b-4712-9ee9-5b3a9b9513d8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:07.457411 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4133f5-958b-4712-9ee9-5b3a9b9513d8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:07.457519 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4133f5-958b-4712-9ee9-5b3a9b9513d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:07.477506 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-0b4133f5-958b-4712-9ee9-5b3a9b9513d8 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:07.478708 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0b4133f5-958b-4712-9ee9-5b3a9b9513d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:07.539658 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1156/2339] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:18:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:07.549709 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ea7b5-061e-4a45-8584-95f0f95dba12 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:07.549998 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ea7b5-061e-4a45-8584-95f0f95dba12 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:07.550277 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ea7b5-061e-4a45-8584-95f0f95dba12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:07.569671 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-f86ea7b5-061e-4a45-8584-95f0f95dba12 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:07.570866 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-f86ea7b5-061e-4a45-8584-95f0f95dba12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:07.579231 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1184/2340] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:18:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:07.583383 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4133f5-958b-4712-9ee9-5b3a9b9513d8 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:07.593654 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4133f5-958b-4712-9ee9-5b3a9b9513d8 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:07.598143 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce657c56-56b3-40dd-84fc-5d693dda2d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:07.598744 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce657c56-56b3-40dd-84fc-5d693dda2d05 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:07.599027 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce657c56-56b3-40dd-84fc-5d693dda2d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:07.599299 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce657c56-56b3-40dd-84fc-5d693dda2d05 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:07.599988 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce657c56-56b3-40dd-84fc-5d693dda2d05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:18:07.600291 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce657c56-56b3-40dd-84fc-5d693dda2d05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:07.601906 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce657c56-56b3-40dd-84fc-5d693dda2d05 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:07.613183 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1157/2341] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:18:07.617257 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ea7b5-061e-4a45-8584-95f0f95dba12 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:07.626314 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ea7b5-061e-4a45-8584-95f0f95dba12 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:07.631117 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc379b00-e66b-4408-84f2-76d386dea5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:07.631668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc379b00-e66b-4408-84f2-76d386dea5fa admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:07.631948 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc379b00-e66b-4408-84f2-76d386dea5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:07.632222 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc379b00-e66b-4408-84f2-76d386dea5fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:07.632925 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc379b00-e66b-4408-84f2-76d386dea5fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:07.634815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc379b00-e66b-4408-84f2-76d386dea5fa admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:07.662444 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1185/2342] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:18:07 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:18:07.666726 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce657c56-56b3-40dd-84fc-5d693dda2d05 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:07.677107 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce657c56-56b3-40dd-84fc-5d693dda2d05 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:07.703038 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:07.734611 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c1170-9ca3-4502-b428-2c8d7abbefe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:07.735218 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c1170-9ca3-4502-b428-2c8d7abbefe4 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:07.735499 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c1170-9ca3-4502-b428-2c8d7abbefe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:07.735800 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c1170-9ca3-4502-b428-2c8d7abbefe4 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:07.736474 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9c1170-9ca3-4502-b428-2c8d7abbefe4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:07.738377 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9c1170-9ca3-4502-b428-2c8d7abbefe4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:07.760022 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1158/2343] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:18:07 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:18:07.764130 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc379b00-e66b-4408-84f2-76d386dea5fa admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:07.774644 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc379b00-e66b-4408-84f2-76d386dea5fa admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:07.779221 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5557c647-b4b6-4174-b56b-cff02648f9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:07.779825 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557c647-b4b6-4174-b56b-cff02648f9b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:07.780111 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557c647-b4b6-4174-b56b-cff02648f9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:07.780387 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557c647-b4b6-4174-b56b-cff02648f9b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:07.781111 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5557c647-b4b6-4174-b56b-cff02648f9b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:07.782710 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5557c647-b4b6-4174-b56b-cff02648f9b1 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:07.792453 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1186/2344] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:18:07.796817 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c1170-9ca3-4502-b428-2c8d7abbefe4 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:07.806854 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c1170-9ca3-4502-b428-2c8d7abbefe4 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:07.812449 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86478aa2-09bb-4658-bd53-5558cc9a06f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:07.813111 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86478aa2-09bb-4658-bd53-5558cc9a06f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:07.813398 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86478aa2-09bb-4658-bd53-5558cc9a06f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:07.813679 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86478aa2-09bb-4658-bd53-5558cc9a06f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:07.814377 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86478aa2-09bb-4658-bd53-5558cc9a06f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:07.816002 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86478aa2-09bb-4658-bd53-5558cc9a06f8 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:07.827005 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1159/2345] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:18:07.831217 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5557c647-b4b6-4174-b56b-cff02648f9b1 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:07.841751 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5557c647-b4b6-4174-b56b-cff02648f9b1 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:07.846269 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb99d72-80ee-42ec-aa3c-c8b836026fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:07.846922 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb99d72-80ee-42ec-aa3c-c8b836026fd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:07.847206 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb99d72-80ee-42ec-aa3c-c8b836026fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:07.847482 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb99d72-80ee-42ec-aa3c-c8b836026fd9 admin admin] PATH_INFO: `/v3/projects/3370a01d92f746d59e33805400462a0e/users/85d8bafee1e547c985d23ffd363533c9/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:07.862705 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb99d72-80ee-42ec-aa3c-c8b836026fd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3370a01d92f746d59e33805400462a0e, user_id=85d8bafee1e547c985d23ffd363533c9, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:07.865374 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb99d72-80ee-42ec-aa3c-c8b836026fd9 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:07.878374 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1187/2346] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:18:07 2022] PUT /identity/v3/projects/3370a01d92f746d59e33805400462a0e/users/85d8bafee1e547c985d23ffd363533c9/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:07.888451 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0a216-9bc7-43b2-8676-be7a27567734 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:07.888747 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0a216-9bc7-43b2-8676-be7a27567734 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:07.889077 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0a216-9bc7-43b2-8676-be7a27567734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:07.916338 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-72a0a216-9bc7-43b2-8676-be7a27567734 None None] MFA Rules not processed for user `85d8bafee1e547c985d23ffd363533c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:07.917653 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-72a0a216-9bc7-43b2-8676-be7a27567734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:07.997686 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1160/2347] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:18:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:08.001577 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb99d72-80ee-42ec-aa3c-c8b836026fd9 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:08.023552 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:08.068209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb99d72-80ee-42ec-aa3c-c8b836026fd9 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:08.073099 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d694c-1341-488a-ae31-db55fac8bdd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:08.073710 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d694c-1341-488a-ae31-db55fac8bdd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:08.073998 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d694c-1341-488a-ae31-db55fac8bdd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:08.074272 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d694c-1341-488a-ae31-db55fac8bdd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:08.074964 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d694c-1341-488a-ae31-db55fac8bdd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:08.076904 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d694c-1341-488a-ae31-db55fac8bdd1 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:08.102311 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1188/2348] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:18:08 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:18:08.107036 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0a216-9bc7-43b2-8676-be7a27567734 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:08.117732 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0a216-9bc7-43b2-8676-be7a27567734 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:08.173286 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b779c6-397a-4b61-acbd-e9e0747409c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:08.173905 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b779c6-397a-4b61-acbd-e9e0747409c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:08.174189 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b779c6-397a-4b61-acbd-e9e0747409c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:08.174650 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b779c6-397a-4b61-acbd-e9e0747409c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:08.175342 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b779c6-397a-4b61-acbd-e9e0747409c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:08.177369 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b779c6-397a-4b61-acbd-e9e0747409c6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:08.198590 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1161/2349] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:18:08 2022] POST /identity/v3/users => generated 297 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:18:08.202836 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d694c-1341-488a-ae31-db55fac8bdd1 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:08.213089 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d694c-1341-488a-ae31-db55fac8bdd1 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:08.217695 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709fb1e-e6f9-4385-b97e-7fc750e80062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:08.218309 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709fb1e-e6f9-4385-b97e-7fc750e80062 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:08.218596 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709fb1e-e6f9-4385-b97e-7fc750e80062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:08.218875 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709fb1e-e6f9-4385-b97e-7fc750e80062 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:08.219575 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d709fb1e-e6f9-4385-b97e-7fc750e80062 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:08.221218 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d709fb1e-e6f9-4385-b97e-7fc750e80062 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:08.232058 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1189/2350] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:18:08.236118 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b779c6-397a-4b61-acbd-e9e0747409c6 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:08.245982 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b779c6-397a-4b61-acbd-e9e0747409c6 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:08.251250 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2b837-0943-4cce-9935-ce2b1ac1de6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:08.251913 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2b837-0943-4cce-9935-ce2b1ac1de6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:08.252202 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2b837-0943-4cce-9935-ce2b1ac1de6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:08.252506 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2b837-0943-4cce-9935-ce2b1ac1de6f admin admin] PATH_INFO: `/v3/projects/3690b7dc07854978816cee9ff1d36280/users/519ade39e92342a09e5a6b3e09001832/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:08.266996 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd2b837-0943-4cce-9935-ce2b1ac1de6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3690b7dc07854978816cee9ff1d36280, user_id=519ade39e92342a09e5a6b3e09001832, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:08.269689 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd2b837-0943-4cce-9935-ce2b1ac1de6f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:08.282418 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1162/2351] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:18:08 2022] PUT /identity/v3/projects/3690b7dc07854978816cee9ff1d36280/users/519ade39e92342a09e5a6b3e09001832/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:08.286332 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709fb1e-e6f9-4385-b97e-7fc750e80062 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:08.296809 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709fb1e-e6f9-4385-b97e-7fc750e80062 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:08.366412 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4bc012-fd70-4b2b-a5d8-f5dc21bf46cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:08.367048 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4bc012-fd70-4b2b-a5d8-f5dc21bf46cf admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:08.367414 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4bc012-fd70-4b2b-a5d8-f5dc21bf46cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:08.367724 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4bc012-fd70-4b2b-a5d8-f5dc21bf46cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:08.368587 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4bc012-fd70-4b2b-a5d8-f5dc21bf46cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:08.370209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4bc012-fd70-4b2b-a5d8-f5dc21bf46cf admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:08.381395 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1190/2352] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:18:08.386886 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2b837-0943-4cce-9935-ce2b1ac1de6f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:08.393274 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:08.400109 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2b837-0943-4cce-9935-ce2b1ac1de6f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:08.406670 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3db5e0-2de7-4c6b-aab5-abe789245c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:08.407499 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3db5e0-2de7-4c6b-aab5-abe789245c40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:08.407781 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3db5e0-2de7-4c6b-aab5-abe789245c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:08.408058 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3db5e0-2de7-4c6b-aab5-abe789245c40 admin admin] PATH_INFO: `/v3/projects/3690b7dc07854978816cee9ff1d36280/users/519ade39e92342a09e5a6b3e09001832/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:08.410190 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3db5e0-2de7-4c6b-aab5-abe789245c40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3690b7dc07854978816cee9ff1d36280, user_id=519ade39e92342a09e5a6b3e09001832, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:08.413323 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3db5e0-2de7-4c6b-aab5-abe789245c40 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:08.423962 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1163/2353] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:18:08 2022] PUT /identity/v3/projects/3690b7dc07854978816cee9ff1d36280/users/519ade39e92342a09e5a6b3e09001832/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:08.434000 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a1a66-7559-4eb4-86a5-c23b75066fbc None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:08.434314 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a1a66-7559-4eb4-86a5-c23b75066fbc None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:08.434600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a1a66-7559-4eb4-86a5-c23b75066fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:08.462968 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-d38a1a66-7559-4eb4-86a5-c23b75066fbc None None] MFA Rules not processed for user `519ade39e92342a09e5a6b3e09001832`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:08.464212 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-d38a1a66-7559-4eb4-86a5-c23b75066fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:08.530577 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1191/2354] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:18:08 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:08.542795 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3db5e0-2de7-4c6b-aab5-abe789245c40 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:08.544583 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-bf3db5e0-2de7-4c6b-aab5-abe789245c40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:08.603639 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3db5e0-2de7-4c6b-aab5-abe789245c40 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:08.607298 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b3f27-0e23-46fe-b408-9d8fffa6fbad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:08.607918 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b3f27-0e23-46fe-b408-9d8fffa6fbad service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:08.608203 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b3f27-0e23-46fe-b408-9d8fffa6fbad service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:08.608508 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b3f27-0e23-46fe-b408-9d8fffa6fbad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:08.617290 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8b3f27-0e23-46fe-b408-9d8fffa6fbad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:08.618715 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8b3f27-0e23-46fe-b408-9d8fffa6fbad service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:08.628886 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1164/2355] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:18:08 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:18:08.942634 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a1a66-7559-4eb4-86a5-c23b75066fbc None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:08.952499 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a1a66-7559-4eb4-86a5-c23b75066fbc None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:08.957066 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4825b-c985-44b3-9b06-19302447c2a3 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '519ade39e92342a09e5a6b3e09001832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3690b7dc07854978816cee9ff1d36280', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:08.957681 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4825b-c985-44b3-9b06-19302447c2a3 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:08.957971 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4825b-c985-44b3-9b06-19302447c2a3 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:08.958259 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4825b-c985-44b3-9b06-19302447c2a3 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:08.958966 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced4825b-c985-44b3-9b06-19302447c2a3 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:18:08.959279 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced4825b-c985-44b3-9b06-19302447c2a3 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:08.960918 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced4825b-c985-44b3-9b06-19302447c2a3 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:08.973294 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1192/2356] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:18:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:18:08.977231 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b3f27-0e23-46fe-b408-9d8fffa6fbad service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:08.986181 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b3f27-0e23-46fe-b408-9d8fffa6fbad service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:08.990699 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db21997-aeff-4107-905c-824b140809e8 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '519ade39e92342a09e5a6b3e09001832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3690b7dc07854978816cee9ff1d36280', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:08.991304 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db21997-aeff-4107-905c-824b140809e8 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:08.991589 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db21997-aeff-4107-905c-824b140809e8 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:08.991865 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db21997-aeff-4107-905c-824b140809e8 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:08.992549 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db21997-aeff-4107-905c-824b140809e8 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:08.994460 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db21997-aeff-4107-905c-824b140809e8 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:09.021018 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1165/2357] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:18:08 2022] POST /identity/v3/projects => generated 354 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:18:09.025010 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4825b-c985-44b3-9b06-19302447c2a3 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:09.030997 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:09.037300 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4825b-c985-44b3-9b06-19302447c2a3 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:09.095112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb6b293-9031-461e-aa52-0e93aa351043 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '519ade39e92342a09e5a6b3e09001832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3690b7dc07854978816cee9ff1d36280', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:09.095739 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6b293-9031-461e-aa52-0e93aa351043 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:09.096025 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6b293-9031-461e-aa52-0e93aa351043 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:09.096308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6b293-9031-461e-aa52-0e93aa351043 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:09.097115 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb6b293-9031-461e-aa52-0e93aa351043 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:18:09.097427 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb6b293-9031-461e-aa52-0e93aa351043 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:09.099023 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb6b293-9031-461e-aa52-0e93aa351043 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:09.111189 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1193/2358] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:18:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:18:09.147826 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db21997-aeff-4107-905c-824b140809e8 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:09.158182 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db21997-aeff-4107-905c-824b140809e8 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:09.162813 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e604d1c1-7b2c-449b-bca9-3ad797d2d8b1 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '519ade39e92342a09e5a6b3e09001832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3690b7dc07854978816cee9ff1d36280', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:09.163453 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604d1c1-7b2c-449b-bca9-3ad797d2d8b1 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:09.163744 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604d1c1-7b2c-449b-bca9-3ad797d2d8b1 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:09.164027 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604d1c1-7b2c-449b-bca9-3ad797d2d8b1 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] PATH_INFO: `/v3/projects/20a6009739544aaabf56e3681110d135` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:09.165201 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e604d1c1-7b2c-449b-bca9-3ad797d2d8b1 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] RBAC: Authorizing `identity:delete_project(project_id=20a6009739544aaabf56e3681110d135)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:09.167101 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e604d1c1-7b2c-449b-bca9-3ad797d2d8b1 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:09.214235 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e604d1c1-7b2c-449b-bca9-3ad797d2d8b1 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20a6009739544aaabf56e3681110d135', 'request_id': 'req-e604d1c1-7b2c-449b-bca9-3ad797d2d8b1'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:09.221590 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e604d1c1-7b2c-449b-bca9-3ad797d2d8b1 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20a6009739544aaabf56e3681110d135', 'request_id': 'req-e604d1c1-7b2c-449b-bca9-3ad797d2d8b1'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:09.228280 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e604d1c1-7b2c-449b-bca9-3ad797d2d8b1 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20a6009739544aaabf56e3681110d135', 'request_id': 'req-e604d1c1-7b2c-449b-bca9-3ad797d2d8b1'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:09.238841 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e604d1c1-7b2c-449b-bca9-3ad797d2d8b1 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] The token cache is being invalidate because project 20a6009739544aaabf56e3681110d135 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:18:09.239156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e604d1c1-7b2c-449b-bca9-3ad797d2d8b1 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:09.239792 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e604d1c1-7b2c-449b-bca9-3ad797d2d8b1 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:09.241313 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1166/2359] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:18:09 2022] DELETE /identity/v3/projects/20a6009739544aaabf56e3681110d135 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:09.429530 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb6b293-9031-461e-aa52-0e93aa351043 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:09.431308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-adb6b293-9031-461e-aa52-0e93aa351043 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:09.499410 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb6b293-9031-461e-aa52-0e93aa351043 tempest-VolumeQuotaClassesTest-1225753786 tempest-VolumeQuotaClassesTest-1225753786-project-admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:09.553108 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1422f1-801f-48d9-94ca-79e004bd0797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:09.553746 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1422f1-801f-48d9-94ca-79e004bd0797 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:09.554029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1422f1-801f-48d9-94ca-79e004bd0797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:09.554307 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1422f1-801f-48d9-94ca-79e004bd0797 admin admin] PATH_INFO: `/v3/users/85d8bafee1e547c985d23ffd363533c9` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:09.555485 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1422f1-801f-48d9-94ca-79e004bd0797 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85d8bafee1e547c985d23ffd363533c9)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:09.557430 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1422f1-801f-48d9-94ca-79e004bd0797 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:09.601783 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:09.603657 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-9c1422f1-801f-48d9-94ca-79e004bd0797 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85d8bafee1e547c985d23ffd363533c9'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:09.604635 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-9c1422f1-801f-48d9-94ca-79e004bd0797 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85d8bafee1e547c985d23ffd363533c9'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:09.617887 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-9c1422f1-801f-48d9-94ca-79e004bd0797 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85d8bafee1e547c985d23ffd363533c9'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:09.624998 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-9c1422f1-801f-48d9-94ca-79e004bd0797 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85d8bafee1e547c985d23ffd363533c9'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:09.625612 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-9c1422f1-801f-48d9-94ca-79e004bd0797 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85d8bafee1e547c985d23ffd363533c9'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:09.637736 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1194/2360] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:18:09 2022] DELETE /identity/v3/users/85d8bafee1e547c985d23ffd363533c9 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:09.642267 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb1b0d-7fe4-4a82-a8c7-b8a52c482bb7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:09.642745 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb1b0d-7fe4-4a82-a8c7-b8a52c482bb7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:09.642844 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb1b0d-7fe4-4a82-a8c7-b8a52c482bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:09.661651 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-65bb1b0d-7fe4-4a82-a8c7-b8a52c482bb7 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:09.662844 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-65bb1b0d-7fe4-4a82-a8c7-b8a52c482bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:09.711182 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:09.725005 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1167/2361] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:18:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:09.734552 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1422f1-801f-48d9-94ca-79e004bd0797 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:09.735933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-9c1422f1-801f-48d9-94ca-79e004bd0797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:09.797406 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1422f1-801f-48d9-94ca-79e004bd0797 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:09.801022 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b28d9-78c9-4725-abff-af31e58f016f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:09.801823 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b28d9-78c9-4725-abff-af31e58f016f service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:09.802136 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b28d9-78c9-4725-abff-af31e58f016f service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:09.802413 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b28d9-78c9-4725-abff-af31e58f016f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:09.811974 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6b28d9-78c9-4725-abff-af31e58f016f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:09.813652 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6b28d9-78c9-4725-abff-af31e58f016f service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:09.825185 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1195/2362] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:18:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:18:10.096593 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb1b0d-7fe4-4a82-a8c7-b8a52c482bb7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:10.098127 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-65bb1b0d-7fe4-4a82-a8c7-b8a52c482bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:10.111322 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb1b0d-7fe4-4a82-a8c7-b8a52c482bb7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:10.116531 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adecd25-a985-4533-ad49-b6cb7018ed8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:10.117174 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adecd25-a985-4533-ad49-b6cb7018ed8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:10.117493 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adecd25-a985-4533-ad49-b6cb7018ed8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:10.117795 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adecd25-a985-4533-ad49-b6cb7018ed8a admin admin] PATH_INFO: `/v3/projects/3370a01d92f746d59e33805400462a0e` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:10.118969 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adecd25-a985-4533-ad49-b6cb7018ed8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3370a01d92f746d59e33805400462a0e)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:10.120953 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adecd25-a985-4533-ad49-b6cb7018ed8a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:10.167029 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6adecd25-a985-4533-ad49-b6cb7018ed8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3370a01d92f746d59e33805400462a0e', 'request_id': 'req-6adecd25-a985-4533-ad49-b6cb7018ed8a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:10.174963 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6adecd25-a985-4533-ad49-b6cb7018ed8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3370a01d92f746d59e33805400462a0e', 'request_id': 'req-6adecd25-a985-4533-ad49-b6cb7018ed8a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:10.181369 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6adecd25-a985-4533-ad49-b6cb7018ed8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3370a01d92f746d59e33805400462a0e', 'request_id': 'req-6adecd25-a985-4533-ad49-b6cb7018ed8a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:10.191576 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6adecd25-a985-4533-ad49-b6cb7018ed8a admin admin] The token cache is being invalidate because project 3370a01d92f746d59e33805400462a0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:18:10.191866 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6adecd25-a985-4533-ad49-b6cb7018ed8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:10.192507 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6adecd25-a985-4533-ad49-b6cb7018ed8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:10.194258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1168/2363] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:18:10 2022] DELETE /identity/v3/projects/3370a01d92f746d59e33805400462a0e => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:10.198284 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b28d9-78c9-4725-abff-af31e58f016f service neutron] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:10.207824 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b28d9-78c9-4725-abff-af31e58f016f service neutron] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:10.263908 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d14297a-9280-43ee-b558-24e22d5a7953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:10.264572 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14297a-9280-43ee-b558-24e22d5a7953 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:10.264863 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14297a-9280-43ee-b558-24e22d5a7953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:10.265142 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14297a-9280-43ee-b558-24e22d5a7953 admin admin] PATH_INFO: `/v3/users/519ade39e92342a09e5a6b3e09001832` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:10.266295 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d14297a-9280-43ee-b558-24e22d5a7953 admin admin] RBAC: Authorizing `identity:delete_user(user_id=519ade39e92342a09e5a6b3e09001832)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:10.268215 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d14297a-9280-43ee-b558-24e22d5a7953 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:10.314831 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-5d14297a-9280-43ee-b558-24e22d5a7953 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '519ade39e92342a09e5a6b3e09001832'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:10.315477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-5d14297a-9280-43ee-b558-24e22d5a7953 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '519ade39e92342a09e5a6b3e09001832'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:10.327802 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-5d14297a-9280-43ee-b558-24e22d5a7953 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '519ade39e92342a09e5a6b3e09001832'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:10.330262 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:10.335697 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-5d14297a-9280-43ee-b558-24e22d5a7953 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '519ade39e92342a09e5a6b3e09001832'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:10.336690 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-5d14297a-9280-43ee-b558-24e22d5a7953 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '519ade39e92342a09e5a6b3e09001832'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:10.349418 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1196/2364] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:18:10 2022] DELETE /identity/v3/users/519ade39e92342a09e5a6b3e09001832 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:10.625732 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adecd25-a985-4533-ad49-b6cb7018ed8a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:10.627162 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6adecd25-a985-4533-ad49-b6cb7018ed8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:10.640764 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adecd25-a985-4533-ad49-b6cb7018ed8a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:10.698887 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053271b2-4100-4e5e-8458-ee6f6e9e5423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:10.699561 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053271b2-4100-4e5e-8458-ee6f6e9e5423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:10.699844 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053271b2-4100-4e5e-8458-ee6f6e9e5423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:10.700122 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053271b2-4100-4e5e-8458-ee6f6e9e5423 admin admin] PATH_INFO: `/v3/projects/3690b7dc07854978816cee9ff1d36280` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:10.701307 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053271b2-4100-4e5e-8458-ee6f6e9e5423 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3690b7dc07854978816cee9ff1d36280)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:10.703212 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053271b2-4100-4e5e-8458-ee6f6e9e5423 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:10.750780 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-053271b2-4100-4e5e-8458-ee6f6e9e5423 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3690b7dc07854978816cee9ff1d36280', 'request_id': 'req-053271b2-4100-4e5e-8458-ee6f6e9e5423'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:10.759097 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-053271b2-4100-4e5e-8458-ee6f6e9e5423 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3690b7dc07854978816cee9ff1d36280', 'request_id': 'req-053271b2-4100-4e5e-8458-ee6f6e9e5423'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:10.765339 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-053271b2-4100-4e5e-8458-ee6f6e9e5423 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3690b7dc07854978816cee9ff1d36280', 'request_id': 'req-053271b2-4100-4e5e-8458-ee6f6e9e5423'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:10.775480 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-053271b2-4100-4e5e-8458-ee6f6e9e5423 admin admin] The token cache is being invalidate because project 3690b7dc07854978816cee9ff1d36280 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:18:10.775770 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-053271b2-4100-4e5e-8458-ee6f6e9e5423 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:10.776402 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-053271b2-4100-4e5e-8458-ee6f6e9e5423 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:10.777945 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1169/2365] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:18:10 2022] DELETE /identity/v3/projects/3690b7dc07854978816cee9ff1d36280 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:10.783048 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa113265-4b72-40b2-ab19-3ad9aab2042d None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:10.783558 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa113265-4b72-40b2-ab19-3ad9aab2042d None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:10.783664 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa113265-4b72-40b2-ab19-3ad9aab2042d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:10.803459 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-aa113265-4b72-40b2-ab19-3ad9aab2042d None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:10.804720 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-aa113265-4b72-40b2-ab19-3ad9aab2042d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:10.864542 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1197/2366] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:18:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:10.872295 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa25502-ec14-49c4-a8cc-cd0152c7eccf None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:10.872552 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa25502-ec14-49c4-a8cc-cd0152c7eccf None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:10.872835 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa25502-ec14-49c4-a8cc-cd0152c7eccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:10.886893 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:10.892414 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-3aa25502-ec14-49c4-a8cc-cd0152c7eccf None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:10.894163 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3aa25502-ec14-49c4-a8cc-cd0152c7eccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:10.902328 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1170/2367] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:18:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:10.906836 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa113265-4b72-40b2-ab19-3ad9aab2042d None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:10.916356 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa113265-4b72-40b2-ab19-3ad9aab2042d None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:10.921592 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e889581-f09e-4406-a7ae-f61e451dceb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:10.922250 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e889581-f09e-4406-a7ae-f61e451dceb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:10.922532 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e889581-f09e-4406-a7ae-f61e451dceb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:10.922805 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e889581-f09e-4406-a7ae-f61e451dceb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:10.923500 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e889581-f09e-4406-a7ae-f61e451dceb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:18:10.923805 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e889581-f09e-4406-a7ae-f61e451dceb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:10.925433 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e889581-f09e-4406-a7ae-f61e451dceb3 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:10.936741 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1198/2368] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:18:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:18:10.940711 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa25502-ec14-49c4-a8cc-cd0152c7eccf None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:10.950024 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa25502-ec14-49c4-a8cc-cd0152c7eccf None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:10.954598 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7309b0-bf60-4a9e-b44a-57537b1e02de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:10.955197 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7309b0-bf60-4a9e-b44a-57537b1e02de admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:10.955476 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7309b0-bf60-4a9e-b44a-57537b1e02de admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:10.955749 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7309b0-bf60-4a9e-b44a-57537b1e02de admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:10.956414 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7309b0-bf60-4a9e-b44a-57537b1e02de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:10.958319 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7309b0-bf60-4a9e-b44a-57537b1e02de admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:10.987344 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1171/2369] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:18:10 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:18:10.990344 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e889581-f09e-4406-a7ae-f61e451dceb3 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:11.000245 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e889581-f09e-4406-a7ae-f61e451dceb3 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:11.055663 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe84d098-e8f1-47c8-9a66-1886b05eb958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:11.056283 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84d098-e8f1-47c8-9a66-1886b05eb958 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:11.056601 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84d098-e8f1-47c8-9a66-1886b05eb958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:11.056883 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84d098-e8f1-47c8-9a66-1886b05eb958 admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:11.057554 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe84d098-e8f1-47c8-9a66-1886b05eb958 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:11.059534 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe84d098-e8f1-47c8-9a66-1886b05eb958 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:11.080422 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1199/2370] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:18:10 2022] POST /identity/v3/users => generated 284 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:18:11.084685 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7309b0-bf60-4a9e-b44a-57537b1e02de admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:11.095369 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7309b0-bf60-4a9e-b44a-57537b1e02de admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:11.099992 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f188a4ea-b094-42ee-ac1b-cbfdc6574e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:11.100633 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188a4ea-b094-42ee-ac1b-cbfdc6574e39 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:11.100923 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188a4ea-b094-42ee-ac1b-cbfdc6574e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:11.101207 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188a4ea-b094-42ee-ac1b-cbfdc6574e39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:11.101902 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f188a4ea-b094-42ee-ac1b-cbfdc6574e39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:11.103496 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f188a4ea-b094-42ee-ac1b-cbfdc6574e39 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:11.114712 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1172/2371] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:18:11.118601 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe84d098-e8f1-47c8-9a66-1886b05eb958 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:11.129155 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe84d098-e8f1-47c8-9a66-1886b05eb958 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:11.133864 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2527bf-b7f1-4b39-af14-d5b3fa0d7a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:11.134478 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2527bf-b7f1-4b39-af14-d5b3fa0d7a11 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:11.134793 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2527bf-b7f1-4b39-af14-d5b3fa0d7a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:11.135072 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2527bf-b7f1-4b39-af14-d5b3fa0d7a11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:11.135771 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2527bf-b7f1-4b39-af14-d5b3fa0d7a11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:11.138112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2527bf-b7f1-4b39-af14-d5b3fa0d7a11 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:11.148923 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1200/2372] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:18:11.152749 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f188a4ea-b094-42ee-ac1b-cbfdc6574e39 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:11.162909 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f188a4ea-b094-42ee-ac1b-cbfdc6574e39 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:11.167531 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ac9ef-0369-47bf-b4ff-6acb5f011aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:11.168194 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ac9ef-0369-47bf-b4ff-6acb5f011aac admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:11.168503 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ac9ef-0369-47bf-b4ff-6acb5f011aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:11.168791 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ac9ef-0369-47bf-b4ff-6acb5f011aac admin admin] PATH_INFO: `/v3/projects/e8fe6d4afcd4470491985c3ecc260960/users/ccd005a7e27e44e591a2dca64af6158a/roles/926b9f36c24847c582c84af904250abc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:11.183067 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967ac9ef-0369-47bf-b4ff-6acb5f011aac admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8fe6d4afcd4470491985c3ecc260960, user_id=ccd005a7e27e44e591a2dca64af6158a, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:11.186311 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967ac9ef-0369-47bf-b4ff-6acb5f011aac admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:11.200217 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1173/2373] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:18:11 2022] PUT /identity/v3/projects/e8fe6d4afcd4470491985c3ecc260960/users/ccd005a7e27e44e591a2dca64af6158a/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:11.210357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a679b8-85e7-4f15-9bb3-60fb6ecf672f None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:11.210645 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a679b8-85e7-4f15-9bb3-60fb6ecf672f None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:11.210924 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a679b8-85e7-4f15-9bb3-60fb6ecf672f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:11.238213 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-69a679b8-85e7-4f15-9bb3-60fb6ecf672f None None] MFA Rules not processed for user `ccd005a7e27e44e591a2dca64af6158a`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:11.239441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-69a679b8-85e7-4f15-9bb3-60fb6ecf672f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:11.278556 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:11.303691 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1201/2374] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:18:11 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:11.308045 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ac9ef-0369-47bf-b4ff-6acb5f011aac admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:11.369879 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ac9ef-0369-47bf-b4ff-6acb5f011aac admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:11.374526 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e658ce-eba6-4d1e-9867-7a72fcc5fd03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:11.375144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e658ce-eba6-4d1e-9867-7a72fcc5fd03 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:11.375429 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e658ce-eba6-4d1e-9867-7a72fcc5fd03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:11.375708 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e658ce-eba6-4d1e-9867-7a72fcc5fd03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:11.376371 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e658ce-eba6-4d1e-9867-7a72fcc5fd03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:11.378300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e658ce-eba6-4d1e-9867-7a72fcc5fd03 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:11.404229 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1174/2375] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:18:11 2022] POST /identity/v3/projects => generated 365 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:18:11.408263 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a679b8-85e7-4f15-9bb3-60fb6ecf672f None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:11.419392 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a679b8-85e7-4f15-9bb3-60fb6ecf672f None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:11.477162 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b2f6ff-5b7d-4c21-a446-bf9b51506329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:11.477781 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2f6ff-5b7d-4c21-a446-bf9b51506329 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:11.478072 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2f6ff-5b7d-4c21-a446-bf9b51506329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:11.478351 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2f6ff-5b7d-4c21-a446-bf9b51506329 admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:11.479024 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b2f6ff-5b7d-4c21-a446-bf9b51506329 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:11.480991 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b2f6ff-5b7d-4c21-a446-bf9b51506329 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:11.501566 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1202/2376] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:18:11 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:18:11.505590 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e658ce-eba6-4d1e-9867-7a72fcc5fd03 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:11.516332 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e658ce-eba6-4d1e-9867-7a72fcc5fd03 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:11.521102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb73d9-0732-4f5a-9ad7-ed8dd5cf266a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:11.521830 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb73d9-0732-4f5a-9ad7-ed8dd5cf266a admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:11.522185 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb73d9-0732-4f5a-9ad7-ed8dd5cf266a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:11.522535 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb73d9-0732-4f5a-9ad7-ed8dd5cf266a admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:11.523325 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfb73d9-0732-4f5a-9ad7-ed8dd5cf266a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:11.524972 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfb73d9-0732-4f5a-9ad7-ed8dd5cf266a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:11.536205 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1175/2377] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:18:11.540341 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b2f6ff-5b7d-4c21-a446-bf9b51506329 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:11.550462 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b2f6ff-5b7d-4c21-a446-bf9b51506329 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:11.555048 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab862f41-90b7-4d08-ae6e-e224081c0fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:11.555706 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab862f41-90b7-4d08-ae6e-e224081c0fcd admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:11.555992 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab862f41-90b7-4d08-ae6e-e224081c0fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:11.556276 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab862f41-90b7-4d08-ae6e-e224081c0fcd admin admin] PATH_INFO: `/v3/projects/33fa21e14d98423d8ea53626712034ff/users/e076ad348a3e46ce997595a8ea013da7/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:11.571389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab862f41-90b7-4d08-ae6e-e224081c0fcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=33fa21e14d98423d8ea53626712034ff, user_id=e076ad348a3e46ce997595a8ea013da7, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:11.574118 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab862f41-90b7-4d08-ae6e-e224081c0fcd admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:11.587268 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1203/2378] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:18:11 2022] PUT /identity/v3/projects/33fa21e14d98423d8ea53626712034ff/users/e076ad348a3e46ce997595a8ea013da7/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:18:11.591328 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb73d9-0732-4f5a-9ad7-ed8dd5cf266a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:11.601378 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb73d9-0732-4f5a-9ad7-ed8dd5cf266a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:11.658381 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eb6572-1fe9-4f68-97b4-5d251a2e16f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:11.658934 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb6572-1fe9-4f68-97b4-5d251a2e16f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:11.659211 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb6572-1fe9-4f68-97b4-5d251a2e16f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:11.659492 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb6572-1fe9-4f68-97b4-5d251a2e16f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:11.660189 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eb6572-1fe9-4f68-97b4-5d251a2e16f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:11.661818 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eb6572-1fe9-4f68-97b4-5d251a2e16f7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:11.671864 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1176/2379] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:18:11.676347 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab862f41-90b7-4d08-ae6e-e224081c0fcd admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:11.687128 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab862f41-90b7-4d08-ae6e-e224081c0fcd admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:11.691745 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18b5db-71b2-4271-b25e-761357c62dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:11.692405 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18b5db-71b2-4271-b25e-761357c62dbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:11.692721 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18b5db-71b2-4271-b25e-761357c62dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:11.693024 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18b5db-71b2-4271-b25e-761357c62dbe admin admin] PATH_INFO: `/v3/projects/33fa21e14d98423d8ea53626712034ff/users/e076ad348a3e46ce997595a8ea013da7/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:11.695224 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba18b5db-71b2-4271-b25e-761357c62dbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=33fa21e14d98423d8ea53626712034ff, user_id=e076ad348a3e46ce997595a8ea013da7, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:11.697884 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba18b5db-71b2-4271-b25e-761357c62dbe admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:11.708283 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1204/2380] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:18:11 2022] PUT /identity/v3/projects/33fa21e14d98423d8ea53626712034ff/users/e076ad348a3e46ce997595a8ea013da7/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:11.775138 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea0de6-d902-433c-92e5-c74182d6cb1a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:11.775711 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea0de6-d902-433c-92e5-c74182d6cb1a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:11.775711 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea0de6-d902-433c-92e5-c74182d6cb1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:11.800969 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:11.804402 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-38ea0de6-d902-433c-92e5-c74182d6cb1a None None] MFA Rules not processed for user `e076ad348a3e46ce997595a8ea013da7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:11.806312 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-38ea0de6-d902-433c-92e5-c74182d6cb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:11.874130 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1177/2381] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:18:11 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:11.885290 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18b5db-71b2-4271-b25e-761357c62dbe admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:11.887674 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-ba18b5db-71b2-4271-b25e-761357c62dbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:11.945335 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18b5db-71b2-4271-b25e-761357c62dbe admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:11.948929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba55598-e9a0-4b0f-b6e9-5d154171ecf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:11.949601 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba55598-e9a0-4b0f-b6e9-5d154171ecf4 service cinder] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:11.949891 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba55598-e9a0-4b0f-b6e9-5d154171ecf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:11.950173 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba55598-e9a0-4b0f-b6e9-5d154171ecf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:11.960131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba55598-e9a0-4b0f-b6e9-5d154171ecf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:11.961617 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba55598-e9a0-4b0f-b6e9-5d154171ecf4 service cinder] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:11.971040 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1205/2382] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:18:11 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:18:12.053744 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ea0de6-d902-433c-92e5-c74182d6cb1a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:12.063612 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ea0de6-d902-433c-92e5-c74182d6cb1a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:12.067176 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b12cf-c8e9-47ea-a32b-70b57f664793 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:12.067814 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b12cf-c8e9-47ea-a32b-70b57f664793 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:12.068366 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b12cf-c8e9-47ea-a32b-70b57f664793 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:12.068676 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b12cf-c8e9-47ea-a32b-70b57f664793 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:12.125978 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64b12cf-c8e9-47ea-a32b-70b57f664793 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:12.127411 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64b12cf-c8e9-47ea-a32b-70b57f664793 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:12.136186 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1178/2383] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:18:12 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:18:16.979363 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba55598-e9a0-4b0f-b6e9-5d154171ecf4 service cinder] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:17.018040 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:17.043631 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba55598-e9a0-4b0f-b6e9-5d154171ecf4 service cinder] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:17.048125 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a2868d-ab5b-4c47-8b58-263ec24eea08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:17.048803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a2868d-ab5b-4c47-8b58-263ec24eea08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:17.049095 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a2868d-ab5b-4c47-8b58-263ec24eea08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:17.049375 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a2868d-ab5b-4c47-8b58-263ec24eea08 admin admin] PATH_INFO: `/v3/users/ccd005a7e27e44e591a2dca64af6158a` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:17.050543 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a2868d-ab5b-4c47-8b58-263ec24eea08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccd005a7e27e44e591a2dca64af6158a)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:17.052453 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a2868d-ab5b-4c47-8b58-263ec24eea08 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:17.098053 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-21a2868d-ab5b-4c47-8b58-263ec24eea08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccd005a7e27e44e591a2dca64af6158a'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:17.098624 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-21a2868d-ab5b-4c47-8b58-263ec24eea08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccd005a7e27e44e591a2dca64af6158a'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:17.110836 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-21a2868d-ab5b-4c47-8b58-263ec24eea08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccd005a7e27e44e591a2dca64af6158a'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:17.117361 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-21a2868d-ab5b-4c47-8b58-263ec24eea08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccd005a7e27e44e591a2dca64af6158a'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:17.118125 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-21a2868d-ab5b-4c47-8b58-263ec24eea08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccd005a7e27e44e591a2dca64af6158a'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:17.130641 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1206/2384] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:18:16 2022] DELETE /identity/v3/users/ccd005a7e27e44e591a2dca64af6158a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:17.135155 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b9261-1718-490c-9251-fbf30f58d2ab None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:17.135610 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b9261-1718-490c-9251-fbf30f58d2ab None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:17.135756 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b9261-1718-490c-9251-fbf30f58d2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:17.155249 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ff1b9261-1718-490c-9251-fbf30f58d2ab None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:17.156458 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ff1b9261-1718-490c-9251-fbf30f58d2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:17.215181 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1179/2385] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:18:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:17.224865 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a2868d-ab5b-4c47-8b58-263ec24eea08 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:17.226253 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-21a2868d-ab5b-4c47-8b58-263ec24eea08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:17.282937 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a2868d-ab5b-4c47-8b58-263ec24eea08 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:17.286547 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19a752-c1a3-456b-a39a-77a9b25af878 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:17.287174 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19a752-c1a3-456b-a39a-77a9b25af878 service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:17.287443 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19a752-c1a3-456b-a39a-77a9b25af878 service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:17.287719 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19a752-c1a3-456b-a39a-77a9b25af878 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:17.296584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c19a752-c1a3-456b-a39a-77a9b25af878 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:17.298000 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c19a752-c1a3-456b-a39a-77a9b25af878 service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:17.307267 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1207/2386] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:18:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:18:17.590169 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b9261-1718-490c-9251-fbf30f58d2ab None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:17.592213 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ff1b9261-1718-490c-9251-fbf30f58d2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:17.604255 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b9261-1718-490c-9251-fbf30f58d2ab None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:17.609478 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761073f5-f9dc-4db3-a1c0-c9de3c62831c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:17.610136 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761073f5-f9dc-4db3-a1c0-c9de3c62831c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:17.610398 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761073f5-f9dc-4db3-a1c0-c9de3c62831c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:17.610675 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761073f5-f9dc-4db3-a1c0-c9de3c62831c admin admin] PATH_INFO: `/v3/projects/e8fe6d4afcd4470491985c3ecc260960` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:17.611933 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761073f5-f9dc-4db3-a1c0-c9de3c62831c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8fe6d4afcd4470491985c3ecc260960)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:17.613923 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761073f5-f9dc-4db3-a1c0-c9de3c62831c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:17.649940 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:17.663227 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-761073f5-f9dc-4db3-a1c0-c9de3c62831c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8fe6d4afcd4470491985c3ecc260960', 'request_id': 'req-761073f5-f9dc-4db3-a1c0-c9de3c62831c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:17.670707 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-761073f5-f9dc-4db3-a1c0-c9de3c62831c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8fe6d4afcd4470491985c3ecc260960', 'request_id': 'req-761073f5-f9dc-4db3-a1c0-c9de3c62831c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:17.677182 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-761073f5-f9dc-4db3-a1c0-c9de3c62831c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8fe6d4afcd4470491985c3ecc260960', 'request_id': 'req-761073f5-f9dc-4db3-a1c0-c9de3c62831c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:17.688041 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-761073f5-f9dc-4db3-a1c0-c9de3c62831c admin admin] The token cache is being invalidate because project e8fe6d4afcd4470491985c3ecc260960 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:18:17.688334 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-761073f5-f9dc-4db3-a1c0-c9de3c62831c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:17.689010 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-761073f5-f9dc-4db3-a1c0-c9de3c62831c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:17.690515 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1180/2387] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:18:17 2022] DELETE /identity/v3/projects/e8fe6d4afcd4470491985c3ecc260960 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:17.694970 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19a752-c1a3-456b-a39a-77a9b25af878 service neutron] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:17.705787 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19a752-c1a3-456b-a39a-77a9b25af878 service neutron] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:17.755758 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:17.762505 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a179ff62-b39f-481b-9944-57e501c12bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:17.763171 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179ff62-b39f-481b-9944-57e501c12bec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:17.763770 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179ff62-b39f-481b-9944-57e501c12bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:17.764168 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179ff62-b39f-481b-9944-57e501c12bec admin admin] PATH_INFO: `/v3/users/e076ad348a3e46ce997595a8ea013da7` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:17.765559 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a179ff62-b39f-481b-9944-57e501c12bec admin admin] RBAC: Authorizing `identity:delete_user(user_id=e076ad348a3e46ce997595a8ea013da7)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:17.767965 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a179ff62-b39f-481b-9944-57e501c12bec admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:17.814536 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-a179ff62-b39f-481b-9944-57e501c12bec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e076ad348a3e46ce997595a8ea013da7'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:17.815204 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-a179ff62-b39f-481b-9944-57e501c12bec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e076ad348a3e46ce997595a8ea013da7'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:17.828670 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-a179ff62-b39f-481b-9944-57e501c12bec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e076ad348a3e46ce997595a8ea013da7'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:17.835973 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-a179ff62-b39f-481b-9944-57e501c12bec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e076ad348a3e46ce997595a8ea013da7'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:17.836718 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-a179ff62-b39f-481b-9944-57e501c12bec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e076ad348a3e46ce997595a8ea013da7'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:17.849152 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1208/2388] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:18:17 2022] DELETE /identity/v3/users/e076ad348a3e46ce997595a8ea013da7 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:18.130268 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761073f5-f9dc-4db3-a1c0-c9de3c62831c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:18.131775 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-761073f5-f9dc-4db3-a1c0-c9de3c62831c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:18.145441 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761073f5-f9dc-4db3-a1c0-c9de3c62831c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:18.203143 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d760b2e-0e9d-49d6-9ce4-e61712b28a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:18.203714 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d760b2e-0e9d-49d6-9ce4-e61712b28a36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:18.204003 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d760b2e-0e9d-49d6-9ce4-e61712b28a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:18.204309 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d760b2e-0e9d-49d6-9ce4-e61712b28a36 admin admin] PATH_INFO: `/v3/projects/33fa21e14d98423d8ea53626712034ff` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:18.205505 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d760b2e-0e9d-49d6-9ce4-e61712b28a36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33fa21e14d98423d8ea53626712034ff)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:18.207426 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d760b2e-0e9d-49d6-9ce4-e61712b28a36 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:18.253924 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9d760b2e-0e9d-49d6-9ce4-e61712b28a36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33fa21e14d98423d8ea53626712034ff', 'request_id': 'req-9d760b2e-0e9d-49d6-9ce4-e61712b28a36'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:18.261231 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9d760b2e-0e9d-49d6-9ce4-e61712b28a36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33fa21e14d98423d8ea53626712034ff', 'request_id': 'req-9d760b2e-0e9d-49d6-9ce4-e61712b28a36'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:18.267318 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9d760b2e-0e9d-49d6-9ce4-e61712b28a36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33fa21e14d98423d8ea53626712034ff', 'request_id': 'req-9d760b2e-0e9d-49d6-9ce4-e61712b28a36'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:18.277470 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9d760b2e-0e9d-49d6-9ce4-e61712b28a36 admin admin] The token cache is being invalidate because project 33fa21e14d98423d8ea53626712034ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:18:18.277762 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9d760b2e-0e9d-49d6-9ce4-e61712b28a36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:18.278395 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9d760b2e-0e9d-49d6-9ce4-e61712b28a36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:18.280082 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1181/2389] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:18:18 2022] DELETE /identity/v3/projects/33fa21e14d98423d8ea53626712034ff => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:18.284992 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a4c5c5-ed19-43e4-9f77-e59ec4dc6fdd None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:18.285469 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a4c5c5-ed19-43e4-9f77-e59ec4dc6fdd None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:18.285566 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a4c5c5-ed19-43e4-9f77-e59ec4dc6fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:18.304949 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-83a4c5c5-ed19-43e4-9f77-e59ec4dc6fdd None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:18.306139 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-83a4c5c5-ed19-43e4-9f77-e59ec4dc6fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:18.364943 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1209/2390] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:18:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:18.372774 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9485a3-072c-45c4-95d7-f7ff9d1a825c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:18.373067 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9485a3-072c-45c4-95d7-f7ff9d1a825c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:18.373346 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9485a3-072c-45c4-95d7-f7ff9d1a825c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:18.392396 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-bd9485a3-072c-45c4-95d7-f7ff9d1a825c None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:18.393615 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-bd9485a3-072c-45c4-95d7-f7ff9d1a825c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:18.401353 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1182/2391] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:18:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:18.405894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a4c5c5-ed19-43e4-9f77-e59ec4dc6fdd None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:18.415467 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a4c5c5-ed19-43e4-9f77-e59ec4dc6fdd None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:18.420626 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12138e0-286a-46e5-b652-6df63a569e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:18.421223 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12138e0-286a-46e5-b652-6df63a569e10 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:18.421502 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12138e0-286a-46e5-b652-6df63a569e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:18.421774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12138e0-286a-46e5-b652-6df63a569e10 admin admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:18.422468 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12138e0-286a-46e5-b652-6df63a569e10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:18:18.422772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12138e0-286a-46e5-b652-6df63a569e10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:18.424354 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12138e0-286a-46e5-b652-6df63a569e10 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:18.436213 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1210/2392] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:18:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:18:18.440543 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9485a3-072c-45c4-95d7-f7ff9d1a825c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:18.449931 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9485a3-072c-45c4-95d7-f7ff9d1a825c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:18.455209 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198cb97a-501f-4fa0-9b06-7cb8a719f19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:18.455757 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198cb97a-501f-4fa0-9b06-7cb8a719f19e admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:18.456043 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198cb97a-501f-4fa0-9b06-7cb8a719f19e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:18.456317 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198cb97a-501f-4fa0-9b06-7cb8a719f19e admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:18.457019 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198cb97a-501f-4fa0-9b06-7cb8a719f19e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:18.458891 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198cb97a-501f-4fa0-9b06-7cb8a719f19e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:18.488443 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1183/2393] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:18:18 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:18:18.490993 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12138e0-286a-46e5-b652-6df63a569e10 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:18.502764 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12138e0-286a-46e5-b652-6df63a569e10 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:18.560672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d06a97b-0083-4dde-957c-0dc4feb1e1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:18.561337 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06a97b-0083-4dde-957c-0dc4feb1e1f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:18.561620 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06a97b-0083-4dde-957c-0dc4feb1e1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:18.561898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06a97b-0083-4dde-957c-0dc4feb1e1f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:18.562590 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d06a97b-0083-4dde-957c-0dc4feb1e1f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:18.564526 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d06a97b-0083-4dde-957c-0dc4feb1e1f4 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:18.575073 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:18.590726 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1211/2394] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:18:18 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:18:18.591934 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198cb97a-501f-4fa0-9b06-7cb8a719f19e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:18.602394 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198cb97a-501f-4fa0-9b06-7cb8a719f19e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:18.607689 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8260b27d-ad13-49ba-9746-a98b42222327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:18.608293 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260b27d-ad13-49ba-9746-a98b42222327 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:18.608614 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260b27d-ad13-49ba-9746-a98b42222327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:18.608893 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260b27d-ad13-49ba-9746-a98b42222327 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:18.609614 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8260b27d-ad13-49ba-9746-a98b42222327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:18.611200 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8260b27d-ad13-49ba-9746-a98b42222327 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:18.621023 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1184/2395] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:18:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:18:18.624852 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d06a97b-0083-4dde-957c-0dc4feb1e1f4 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:18.634687 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d06a97b-0083-4dde-957c-0dc4feb1e1f4 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:18.639248 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e56e9-6303-48f9-b39e-8555957b9e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:18.639847 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e56e9-6303-48f9-b39e-8555957b9e69 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:18.640132 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e56e9-6303-48f9-b39e-8555957b9e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:18.640409 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e56e9-6303-48f9-b39e-8555957b9e69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:18.641136 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e56e9-6303-48f9-b39e-8555957b9e69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:18.642813 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e56e9-6303-48f9-b39e-8555957b9e69 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:18.653669 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1212/2396] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:18:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:18:18.657447 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8260b27d-ad13-49ba-9746-a98b42222327 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:18.667196 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8260b27d-ad13-49ba-9746-a98b42222327 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:18.672435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52587216-49cc-4811-8c24-920a2ebd75f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:18.673115 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52587216-49cc-4811-8c24-920a2ebd75f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:18.673398 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52587216-49cc-4811-8c24-920a2ebd75f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:18.673675 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52587216-49cc-4811-8c24-920a2ebd75f5 admin admin] PATH_INFO: `/v3/projects/d00f7eb589a44ad48b9bb78f0334cb97/users/deb7fff1ca544f20addd03f1ef34b31d/roles/926b9f36c24847c582c84af904250abc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:18.681801 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:18.689683 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52587216-49cc-4811-8c24-920a2ebd75f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d00f7eb589a44ad48b9bb78f0334cb97, user_id=deb7fff1ca544f20addd03f1ef34b31d, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:18.692411 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52587216-49cc-4811-8c24-920a2ebd75f5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:18.705940 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1185/2397] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:18:18 2022] PUT /identity/v3/projects/d00f7eb589a44ad48b9bb78f0334cb97/users/deb7fff1ca544f20addd03f1ef34b31d/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:18:18.716322 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41533c8d-61af-483d-a5de-9b52b8b5a089 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:18.716633 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41533c8d-61af-483d-a5de-9b52b8b5a089 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:18.716910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41533c8d-61af-483d-a5de-9b52b8b5a089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:18.744003 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-41533c8d-61af-483d-a5de-9b52b8b5a089 None None] MFA Rules not processed for user `deb7fff1ca544f20addd03f1ef34b31d`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:18.745240 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-41533c8d-61af-483d-a5de-9b52b8b5a089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:18.810254 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1213/2398] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:18:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:18.821473 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52587216-49cc-4811-8c24-920a2ebd75f5 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:18.823138 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-52587216-49cc-4811-8c24-920a2ebd75f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:18.884224 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52587216-49cc-4811-8c24-920a2ebd75f5 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:18.888432 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f2902-ff64-4524-a176-9082a7ab5b5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:18.889076 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f2902-ff64-4524-a176-9082a7ab5b5a service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:18.889360 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f2902-ff64-4524-a176-9082a7ab5b5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:18.889645 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f2902-ff64-4524-a176-9082a7ab5b5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:18.897785 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76f2902-ff64-4524-a176-9082a7ab5b5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:18.899206 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76f2902-ff64-4524-a176-9082a7ab5b5a service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:18.908439 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1186/2399] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:18:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:18:18.932290 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41533c8d-61af-483d-a5de-9b52b8b5a089 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:18.994680 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41533c8d-61af-483d-a5de-9b52b8b5a089 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:18.999356 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6ddc6-e23a-4bf0-8b42-e1189e9467bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:18.999999 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6ddc6-e23a-4bf0-8b42-e1189e9467bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:19.000294 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6ddc6-e23a-4bf0-8b42-e1189e9467bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:19.000608 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6ddc6-e23a-4bf0-8b42-e1189e9467bf admin admin] PATH_INFO: `/v3/users/deb7fff1ca544f20addd03f1ef34b31d` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:19.001793 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c6ddc6-e23a-4bf0-8b42-e1189e9467bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=deb7fff1ca544f20addd03f1ef34b31d)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:19.003722 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c6ddc6-e23a-4bf0-8b42-e1189e9467bf admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:19.050783 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-f5c6ddc6-e23a-4bf0-8b42-e1189e9467bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'deb7fff1ca544f20addd03f1ef34b31d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:19.051462 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-f5c6ddc6-e23a-4bf0-8b42-e1189e9467bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'deb7fff1ca544f20addd03f1ef34b31d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:19.063776 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-f5c6ddc6-e23a-4bf0-8b42-e1189e9467bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'deb7fff1ca544f20addd03f1ef34b31d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:19.071478 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-f5c6ddc6-e23a-4bf0-8b42-e1189e9467bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'deb7fff1ca544f20addd03f1ef34b31d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:19.072223 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-f5c6ddc6-e23a-4bf0-8b42-e1189e9467bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'deb7fff1ca544f20addd03f1ef34b31d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:19.075068 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:19.088128 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1214/2400] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:18:18 2022] DELETE /identity/v3/users/deb7fff1ca544f20addd03f1ef34b31d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:19.092896 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80a70f-67bd-4b27-a81c-bc9df4f9337f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:19.093193 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80a70f-67bd-4b27-a81c-bc9df4f9337f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:19.093477 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80a70f-67bd-4b27-a81c-bc9df4f9337f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:19.111997 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-fc80a70f-67bd-4b27-a81c-bc9df4f9337f None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:19.113240 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fc80a70f-67bd-4b27-a81c-bc9df4f9337f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:19.172762 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1187/2401] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:18:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:19.182422 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6ddc6-e23a-4bf0-8b42-e1189e9467bf admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:19.184344 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-f5c6ddc6-e23a-4bf0-8b42-e1189e9467bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:19.243348 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6ddc6-e23a-4bf0-8b42-e1189e9467bf admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:19.247093 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59cc4f2-ab3a-4077-9a87-d7d9c0592d08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:19.247660 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59cc4f2-ab3a-4077-9a87-d7d9c0592d08 service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:19.247945 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59cc4f2-ab3a-4077-9a87-d7d9c0592d08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:19.248226 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59cc4f2-ab3a-4077-9a87-d7d9c0592d08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:19.257280 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59cc4f2-ab3a-4077-9a87-d7d9c0592d08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:19.258717 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59cc4f2-ab3a-4077-9a87-d7d9c0592d08 service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:19.268164 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1215/2402] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:18:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:18:19.541498 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80a70f-67bd-4b27-a81c-bc9df4f9337f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:19.543099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fc80a70f-67bd-4b27-a81c-bc9df4f9337f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:19.555524 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80a70f-67bd-4b27-a81c-bc9df4f9337f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:19.560188 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f70c60-a488-42de-9445-4d61a0382ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:19.560873 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f70c60-a488-42de-9445-4d61a0382ce8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:19.561346 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f70c60-a488-42de-9445-4d61a0382ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:19.561462 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f70c60-a488-42de-9445-4d61a0382ce8 admin admin] PATH_INFO: `/v3/projects/d00f7eb589a44ad48b9bb78f0334cb97` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:19.562710 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f70c60-a488-42de-9445-4d61a0382ce8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d00f7eb589a44ad48b9bb78f0334cb97)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:19.564662 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f70c60-a488-42de-9445-4d61a0382ce8 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:19.613810 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b9f70c60-a488-42de-9445-4d61a0382ce8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd00f7eb589a44ad48b9bb78f0334cb97', 'request_id': 'req-b9f70c60-a488-42de-9445-4d61a0382ce8'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:19.621026 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b9f70c60-a488-42de-9445-4d61a0382ce8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd00f7eb589a44ad48b9bb78f0334cb97', 'request_id': 'req-b9f70c60-a488-42de-9445-4d61a0382ce8'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:19.627184 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b9f70c60-a488-42de-9445-4d61a0382ce8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd00f7eb589a44ad48b9bb78f0334cb97', 'request_id': 'req-b9f70c60-a488-42de-9445-4d61a0382ce8'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:19.638060 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b9f70c60-a488-42de-9445-4d61a0382ce8 admin admin] The token cache is being invalidate because project d00f7eb589a44ad48b9bb78f0334cb97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:18:19.638353 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b9f70c60-a488-42de-9445-4d61a0382ce8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:19.638995 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b9f70c60-a488-42de-9445-4d61a0382ce8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:19.641130 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1188/2403] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:18:19 2022] DELETE /identity/v3/projects/d00f7eb589a44ad48b9bb78f0334cb97 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:19.645934 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df212f-1858-42e8-becc-5c4ec3a52dcf None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:19.646408 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df212f-1858-42e8-becc-5c4ec3a52dcf None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:19.646505 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df212f-1858-42e8-becc-5c4ec3a52dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:19.665884 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-b2df212f-1858-42e8-becc-5c4ec3a52dcf None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:19.667077 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-b2df212f-1858-42e8-becc-5c4ec3a52dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:19.724115 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1216/2404] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:18:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:19.733131 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e956ffe-0134-43ca-bbc6-3def10a4a201 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:19.733429 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e956ffe-0134-43ca-bbc6-3def10a4a201 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:19.733712 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e956ffe-0134-43ca-bbc6-3def10a4a201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:19.752873 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-3e956ffe-0134-43ca-bbc6-3def10a4a201 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:19.754087 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3e956ffe-0134-43ca-bbc6-3def10a4a201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:19.762430 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1189/2405] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:18:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:19.766478 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df212f-1858-42e8-becc-5c4ec3a52dcf None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:19.776108 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df212f-1858-42e8-becc-5c4ec3a52dcf None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:19.781249 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8169287-82e8-4d0f-8399-dca3f133d0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:19.781857 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8169287-82e8-4d0f-8399-dca3f133d0db admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:19.782134 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8169287-82e8-4d0f-8399-dca3f133d0db admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:19.782407 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8169287-82e8-4d0f-8399-dca3f133d0db admin admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:19.783124 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8169287-82e8-4d0f-8399-dca3f133d0db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:18:19.783478 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8169287-82e8-4d0f-8399-dca3f133d0db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:19.785061 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8169287-82e8-4d0f-8399-dca3f133d0db admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:19.796567 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1217/2406] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:18:19.800996 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e956ffe-0134-43ca-bbc6-3def10a4a201 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:19.811045 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e956ffe-0134-43ca-bbc6-3def10a4a201 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:19.816057 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab2f4d-7fc3-4803-ae5c-2edfa2b5d242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:19.816676 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab2f4d-7fc3-4803-ae5c-2edfa2b5d242 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:19.817128 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab2f4d-7fc3-4803-ae5c-2edfa2b5d242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:19.817435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab2f4d-7fc3-4803-ae5c-2edfa2b5d242 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:19.818398 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ab2f4d-7fc3-4803-ae5c-2edfa2b5d242 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:19.820353 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ab2f4d-7fc3-4803-ae5c-2edfa2b5d242 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:19.825793 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:19.850195 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1190/2407] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:18:19 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:18:19.854402 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8169287-82e8-4d0f-8399-dca3f133d0db admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:19.865230 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8169287-82e8-4d0f-8399-dca3f133d0db admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:19.921515 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f74734-be37-48ab-8667-41651c4b5f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:19.922133 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f74734-be37-48ab-8667-41651c4b5f46 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:19.922419 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f74734-be37-48ab-8667-41651c4b5f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:19.922699 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f74734-be37-48ab-8667-41651c4b5f46 admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:19.923366 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f74734-be37-48ab-8667-41651c4b5f46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:19.925311 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f74734-be37-48ab-8667-41651c4b5f46 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:19.947694 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1218/2408] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:18:19 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:18:19.952404 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab2f4d-7fc3-4803-ae5c-2edfa2b5d242 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:19.962644 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab2f4d-7fc3-4803-ae5c-2edfa2b5d242 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:19.967297 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00dc2c7-7922-47d4-bdbc-eb715cc3cda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:19.967908 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00dc2c7-7922-47d4-bdbc-eb715cc3cda8 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:19.968250 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00dc2c7-7922-47d4-bdbc-eb715cc3cda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:19.968494 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00dc2c7-7922-47d4-bdbc-eb715cc3cda8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:19.969198 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00dc2c7-7922-47d4-bdbc-eb715cc3cda8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:19.970789 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00dc2c7-7922-47d4-bdbc-eb715cc3cda8 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:19.980674 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1191/2409] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:18:19.984383 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f74734-be37-48ab-8667-41651c4b5f46 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:19.994596 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f74734-be37-48ab-8667-41651c4b5f46 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:19.999922 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce024c1-c28f-47a0-98da-2c44639706f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:20.000551 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce024c1-c28f-47a0-98da-2c44639706f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:20.000835 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce024c1-c28f-47a0-98da-2c44639706f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:20.001113 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce024c1-c28f-47a0-98da-2c44639706f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:20.001800 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce024c1-c28f-47a0-98da-2c44639706f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:20.003373 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce024c1-c28f-47a0-98da-2c44639706f2 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:20.008432 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:20.015197 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1219/2410] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:18:20.019415 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00dc2c7-7922-47d4-bdbc-eb715cc3cda8 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:20.029400 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00dc2c7-7922-47d4-bdbc-eb715cc3cda8 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:20.034034 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ecd9ee-90e4-4880-86fd-8e058d8110c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:20.034634 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ecd9ee-90e4-4880-86fd-8e058d8110c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:20.034919 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ecd9ee-90e4-4880-86fd-8e058d8110c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:20.035197 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ecd9ee-90e4-4880-86fd-8e058d8110c4 admin admin] PATH_INFO: `/v3/projects/da11e095c646477d9cfeb617c19d8877/users/a8a3bd8473d64ea4b62c34bf84b1fcee/roles/926b9f36c24847c582c84af904250abc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:20.049445 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ecd9ee-90e4-4880-86fd-8e058d8110c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da11e095c646477d9cfeb617c19d8877, user_id=a8a3bd8473d64ea4b62c34bf84b1fcee, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:20.052083 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ecd9ee-90e4-4880-86fd-8e058d8110c4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:20.065885 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1192/2411] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:18:20 2022] PUT /identity/v3/projects/da11e095c646477d9cfeb617c19d8877/users/a8a3bd8473d64ea4b62c34bf84b1fcee/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:20.076786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660ef5a-cf75-4956-bff3-befd9f7c5e30 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:20.077085 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660ef5a-cf75-4956-bff3-befd9f7c5e30 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:20.077365 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660ef5a-cf75-4956-bff3-befd9f7c5e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:20.106321 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-2660ef5a-cf75-4956-bff3-befd9f7c5e30 None None] MFA Rules not processed for user `a8a3bd8473d64ea4b62c34bf84b1fcee`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:20.107528 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-2660ef5a-cf75-4956-bff3-befd9f7c5e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:20.174893 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1220/2412] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:18:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:20.186320 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ecd9ee-90e4-4880-86fd-8e058d8110c4 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:20.188028 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-93ecd9ee-90e4-4880-86fd-8e058d8110c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:20.244985 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ecd9ee-90e4-4880-86fd-8e058d8110c4 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:20.249269 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45258606-80be-4517-87f4-567b0ec577d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:20.249878 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45258606-80be-4517-87f4-567b0ec577d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:20.250165 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45258606-80be-4517-87f4-567b0ec577d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:20.250443 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45258606-80be-4517-87f4-567b0ec577d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:20.258583 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45258606-80be-4517-87f4-567b0ec577d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:20.260005 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45258606-80be-4517-87f4-567b0ec577d6 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:20.268726 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1193/2413] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:18:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:18:30.166174 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=32 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:30.264886 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=47,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:30.305032 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2660ef5a-cf75-4956-bff3-befd9f7c5e30 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:30.379910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2660ef5a-cf75-4956-bff3-befd9f7c5e30 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:30.384796 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd6cbf-777a-4c9b-8043-d05b1113d789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:30.385560 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd6cbf-777a-4c9b-8043-d05b1113d789 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:30.385879 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd6cbf-777a-4c9b-8043-d05b1113d789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:30.386165 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd6cbf-777a-4c9b-8043-d05b1113d789 admin admin] PATH_INFO: `/v3/users/a8a3bd8473d64ea4b62c34bf84b1fcee` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:30.387408 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dd6cbf-777a-4c9b-8043-d05b1113d789 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8a3bd8473d64ea4b62c34bf84b1fcee)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:30.389438 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dd6cbf-777a-4c9b-8043-d05b1113d789 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:30.448906 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-c2dd6cbf-777a-4c9b-8043-d05b1113d789 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8a3bd8473d64ea4b62c34bf84b1fcee'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:30.449626 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-c2dd6cbf-777a-4c9b-8043-d05b1113d789 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8a3bd8473d64ea4b62c34bf84b1fcee'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:30.464078 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-c2dd6cbf-777a-4c9b-8043-d05b1113d789 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8a3bd8473d64ea4b62c34bf84b1fcee'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:30.472935 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-c2dd6cbf-777a-4c9b-8043-d05b1113d789 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8a3bd8473d64ea4b62c34bf84b1fcee'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:30.473865 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-c2dd6cbf-777a-4c9b-8043-d05b1113d789 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8a3bd8473d64ea4b62c34bf84b1fcee'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:30.489098 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1221/2414] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:18:30 2022] DELETE /identity/v3/users/a8a3bd8473d64ea4b62c34bf84b1fcee => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:18:30.493457 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf0ad19-7af6-45ff-b5dd-6a413dcaf550 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:30.493751 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf0ad19-7af6-45ff-b5dd-6a413dcaf550 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:30.494040 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf0ad19-7af6-45ff-b5dd-6a413dcaf550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:30.518019 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-2bf0ad19-7af6-45ff-b5dd-6a413dcaf550 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:30.519253 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2bf0ad19-7af6-45ff-b5dd-6a413dcaf550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:30.614392 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1194/2415] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:18:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:30.640253 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd6cbf-777a-4c9b-8043-d05b1113d789 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:30.643235 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-c2dd6cbf-777a-4c9b-8043-d05b1113d789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:30.742500 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd6cbf-777a-4c9b-8043-d05b1113d789 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:30.747231 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab9c151-d636-43f8-a3ad-7ae6f1aea85d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:30.747873 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9c151-d636-43f8-a3ad-7ae6f1aea85d service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:30.748163 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9c151-d636-43f8-a3ad-7ae6f1aea85d service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:30.748444 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9c151-d636-43f8-a3ad-7ae6f1aea85d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:30.760972 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab9c151-d636-43f8-a3ad-7ae6f1aea85d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:30.762474 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab9c151-d636-43f8-a3ad-7ae6f1aea85d service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:30.773699 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1222/2416] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:18:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:18:31.052986 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf0ad19-7af6-45ff-b5dd-6a413dcaf550 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:31.054640 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2bf0ad19-7af6-45ff-b5dd-6a413dcaf550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:31.074545 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf0ad19-7af6-45ff-b5dd-6a413dcaf550 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:31.080782 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03e5da6-0f50-4810-b42d-8b638d8b1252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:31.081445 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e5da6-0f50-4810-b42d-8b638d8b1252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:31.081732 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e5da6-0f50-4810-b42d-8b638d8b1252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:31.082015 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e5da6-0f50-4810-b42d-8b638d8b1252 admin admin] PATH_INFO: `/v3/projects/da11e095c646477d9cfeb617c19d8877` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:31.083211 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03e5da6-0f50-4810-b42d-8b638d8b1252 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da11e095c646477d9cfeb617c19d8877)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:31.085372 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03e5da6-0f50-4810-b42d-8b638d8b1252 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:31.143851 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f03e5da6-0f50-4810-b42d-8b638d8b1252 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da11e095c646477d9cfeb617c19d8877', 'request_id': 'req-f03e5da6-0f50-4810-b42d-8b638d8b1252'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:31.154530 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f03e5da6-0f50-4810-b42d-8b638d8b1252 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da11e095c646477d9cfeb617c19d8877', 'request_id': 'req-f03e5da6-0f50-4810-b42d-8b638d8b1252'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:31.162108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f03e5da6-0f50-4810-b42d-8b638d8b1252 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da11e095c646477d9cfeb617c19d8877', 'request_id': 'req-f03e5da6-0f50-4810-b42d-8b638d8b1252'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:31.175889 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f03e5da6-0f50-4810-b42d-8b638d8b1252 admin admin] The token cache is being invalidate because project da11e095c646477d9cfeb617c19d8877 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:18:31.176228 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f03e5da6-0f50-4810-b42d-8b638d8b1252 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:31.176866 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f03e5da6-0f50-4810-b42d-8b638d8b1252 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:18:31.179540 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1195/2417] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:18:31 2022] DELETE /identity/v3/projects/da11e095c646477d9cfeb617c19d8877 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:18:33.699741 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb17f68-16be-405d-87f8-f848ecf3b46b None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:33.699741 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb17f68-16be-405d-87f8-f848ecf3b46b None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:33.699741 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb17f68-16be-405d-87f8-f848ecf3b46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:33.720371 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-ccb17f68-16be-405d-87f8-f848ecf3b46b None None] MFA Rules not processed for user `ebe4b3455a3d4139a1267da97bd83662`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:33.721618 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-ccb17f68-16be-405d-87f8-f848ecf3b46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:33.738605 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:33.780186 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1223/2418] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:18:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:41.172850 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:42.393257 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b110e-3524-4a86-bf87-78848eefb69a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:42.393904 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b110e-3524-4a86-bf87-78848eefb69a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:42.393904 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b110e-3524-4a86-bf87-78848eefb69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:42.413836 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-324b110e-3524-4a86-bf87-78848eefb69a None None] MFA Rules not processed for user `ca3cd1c2aec94b43aff0034b812a069d`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:42.415057 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-324b110e-3524-4a86-bf87-78848eefb69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:42.473818 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1196/2419] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:18:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:43.776024 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=15 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:18:49.992143 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebea9fb-db4b-4c81-a82b-1b6a37fd49bc None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:49.992761 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebea9fb-db4b-4c81-a82b-1b6a37fd49bc None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:49.992997 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebea9fb-db4b-4c81-a82b-1b6a37fd49bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:50.026011 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-aebea9fb-db4b-4c81-a82b-1b6a37fd49bc None None] MFA Rules not processed for user `5007a2f8e6244b0788f0a554cc572fe5`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:50.027314 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-aebea9fb-db4b-4c81-a82b-1b6a37fd49bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:50.098131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1224/2420] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 14:18:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:50.145897 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324b110e-3524-4a86-bf87-78848eefb69a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:50.147283 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-324b110e-3524-4a86-bf87-78848eefb69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:50.216783 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324b110e-3524-4a86-bf87-78848eefb69a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:50.221618 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b71665-661a-4e27-88bd-768fda5d83ce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2a6d7bd10435ba2c8e552d808b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:50.222264 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b71665-661a-4e27-88bd-768fda5d83ce service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:50.222550 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b71665-661a-4e27-88bd-768fda5d83ce service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:50.222828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b71665-661a-4e27-88bd-768fda5d83ce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:50.231319 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b71665-661a-4e27-88bd-768fda5d83ce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:50.232927 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b71665-661a-4e27-88bd-768fda5d83ce service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:50.240809 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1197/2421] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 14:18:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:18:53.595170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebea9fb-db4b-4c81-a82b-1b6a37fd49bc None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:53.598632 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-aebea9fb-db4b-4c81-a82b-1b6a37fd49bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:53.696372 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebea9fb-db4b-4c81-a82b-1b6a37fd49bc None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:53.705011 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf17657-7261-4728-9898-289eefaba23d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:53.705885 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf17657-7261-4728-9898-289eefaba23d service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:53.706419 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf17657-7261-4728-9898-289eefaba23d service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:53.707064 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf17657-7261-4728-9898-289eefaba23d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:53.709271 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-5cf17657-7261-4728-9898-289eefaba23d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:53.802693 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf17657-7261-4728-9898-289eefaba23d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:53.804143 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf17657-7261-4728-9898-289eefaba23d service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:53.820690 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1225/2422] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:18:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:18:55.952074 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2db2dc-36fc-428a-9e86-f5a0707f4a7a None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:55.952988 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2db2dc-36fc-428a-9e86-f5a0707f4a7a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:55.953450 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2db2dc-36fc-428a-9e86-f5a0707f4a7a None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:55.955266 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1198/2423] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 14:18:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:18:55.963940 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b238ea-5143-4000-9ec8-e76983f81451 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:55.964469 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b238ea-5143-4000-9ec8-e76983f81451 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:55.964656 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b238ea-5143-4000-9ec8-e76983f81451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:56.009977 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-c8b238ea-5143-4000-9ec8-e76983f81451 None None] MFA Rules not processed for user `fb9f698e09d246bcbea855613764fbe1`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:56.011243 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-c8b238ea-5143-4000-9ec8-e76983f81451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:56.018642 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1226/2424] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 14:18:55 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:56.035468 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2db2dc-36fc-428a-9e86-f5a0707f4a7a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:56.039916 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2e2db2dc-36fc-428a-9e86-f5a0707f4a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:56.069556 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2db2dc-36fc-428a-9e86-f5a0707f4a7a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:56.074615 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80badc-26c1-45c2-93c6-4990d73a5cde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:56.075386 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80badc-26c1-45c2-93c6-4990d73a5cde service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:56.075677 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80badc-26c1-45c2-93c6-4990d73a5cde service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:56.076641 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80badc-26c1-45c2-93c6-4990d73a5cde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:56.086868 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de80badc-26c1-45c2-93c6-4990d73a5cde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:56.088312 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de80badc-26c1-45c2-93c6-4990d73a5cde service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:56.099151 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1199/2425] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:18:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:18:56.210949 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212808d-5dd3-49cc-989d-fe9d80339bbd None None] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:56.211274 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212808d-5dd3-49cc-989d-fe9d80339bbd None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:56.211640 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212808d-5dd3-49cc-989d-fe9d80339bbd None None] PATH_INFO: `/` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:56.213760 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1227/2426] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 14:18:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 14:18:56.222138 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08d5cb-9140-4170-bb6f-8160eb7ea3ee None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:56.222637 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08d5cb-9140-4170-bb6f-8160eb7ea3ee None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:56.223093 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08d5cb-9140-4170-bb6f-8160eb7ea3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:56.277078 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-af08d5cb-9140-4170-bb6f-8160eb7ea3ee None None] MFA Rules not processed for user `a246654c783b4cc68d212e91bb05d8c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:18:56.279355 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-af08d5cb-9140-4170-bb6f-8160eb7ea3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:56.294692 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1200/2427] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 14:18:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:18:56.308323 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d212808d-5dd3-49cc-989d-fe9d80339bbd None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:56.325250 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d212808d-5dd3-49cc-989d-fe9d80339bbd None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:56.336403 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e913dcd-9662-4f5f-bee7-96c393ec1983 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:56.337111 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e913dcd-9662-4f5f-bee7-96c393ec1983 service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:56.337400 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e913dcd-9662-4f5f-bee7-96c393ec1983 service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:56.337708 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e913dcd-9662-4f5f-bee7-96c393ec1983 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:56.372521 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e913dcd-9662-4f5f-bee7-96c393ec1983 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:56.374277 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e913dcd-9662-4f5f-bee7-96c393ec1983 service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:56.395010 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1228/2428] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:18:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:18:58.657581 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af08d5cb-9140-4170-bb6f-8160eb7ea3ee None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:18:58.678070 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af08d5cb-9140-4170-bb6f-8160eb7ea3ee None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:18:58.695191 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f91186-c6a5-4313-be05-ad8b54eb92e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:18:58.697173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f91186-c6a5-4313-be05-ad8b54eb92e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:18:58.697669 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f91186-c6a5-4313-be05-ad8b54eb92e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:18:58.697887 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f91186-c6a5-4313-be05-ad8b54eb92e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:18:58.700976 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-86f91186-c6a5-4313-be05-ad8b54eb92e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:18:58.738239 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f91186-c6a5-4313-be05-ad8b54eb92e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:18:58.739834 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f91186-c6a5-4313-be05-ad8b54eb92e7 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:18:58.762825 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1201/2429] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:18:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:19:05.991474 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e913dcd-9662-4f5f-bee7-96c393ec1983 service neutron] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:05.993127 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-3e913dcd-9662-4f5f-bee7-96c393ec1983 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:19:06.021854 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:19:06.066556 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e913dcd-9662-4f5f-bee7-96c393ec1983 service neutron] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:06.071009 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e5f4fd-cea1-4f90-8a6f-a5b26c89031b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39a66f7362964389be039937a786bb0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:06.071643 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5f4fd-cea1-4f90-8a6f-a5b26c89031b service glance] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:06.071924 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5f4fd-cea1-4f90-8a6f-a5b26c89031b service glance] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:06.072197 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5f4fd-cea1-4f90-8a6f-a5b26c89031b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:06.075052 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-07e5f4fd-cea1-4f90-8a6f-a5b26c89031b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:19:06.089528 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e5f4fd-cea1-4f90-8a6f-a5b26c89031b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:06.090971 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e5f4fd-cea1-4f90-8a6f-a5b26c89031b service glance] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:06.102543 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1229/2430] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:19:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:19:06.171585 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f91186-c6a5-4313-be05-ad8b54eb92e7 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:06.180419 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f91186-c6a5-4313-be05-ad8b54eb92e7 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:06.184319 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8817474d-c6d9-4d12-a19b-292a817fcbd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:06.184967 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817474d-c6d9-4d12-a19b-292a817fcbd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:06.185289 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817474d-c6d9-4d12-a19b-292a817fcbd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:06.185569 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817474d-c6d9-4d12-a19b-292a817fcbd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:06.187345 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8817474d-c6d9-4d12-a19b-292a817fcbd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:19:06.197883 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8817474d-c6d9-4d12-a19b-292a817fcbd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:06.199354 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8817474d-c6d9-4d12-a19b-292a817fcbd1 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:06.207236 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1202/2431] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:19:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:19:06.280025 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e5f4fd-cea1-4f90-8a6f-a5b26c89031b service glance] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:06.289414 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e5f4fd-cea1-4f90-8a6f-a5b26c89031b service glance] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:06.293082 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7596c1c2-d3e4-4eed-bbd5-a1454ab19dc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:06.293698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7596c1c2-d3e4-4eed-bbd5-a1454ab19dc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:06.293982 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7596c1c2-d3e4-4eed-bbd5-a1454ab19dc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:06.294261 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7596c1c2-d3e4-4eed-bbd5-a1454ab19dc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:06.296003 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-7596c1c2-d3e4-4eed-bbd5-a1454ab19dc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:19:06.309773 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7596c1c2-d3e4-4eed-bbd5-a1454ab19dc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:06.311189 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7596c1c2-d3e4-4eed-bbd5-a1454ab19dc8 service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:06.322952 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1230/2432] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:19:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:19:12.306650 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b11a4-4857-41d8-9ca9-78366a43f2bf None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:12.307097 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b11a4-4857-41d8-9ca9-78366a43f2bf None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:12.307356 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b11a4-4857-41d8-9ca9-78366a43f2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:12.314978 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=85 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:19:12.330108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-9b9b11a4-4857-41d8-9ca9-78366a43f2bf None None] MFA Rules not processed for user `5007a2f8e6244b0788f0a554cc572fe5`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:19:12.331318 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9b9b11a4-4857-41d8-9ca9-78366a43f2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:19:12.338697 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1203/2433] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 14:19:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:19:12.350934 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7596c1c2-d3e4-4eed-bbd5-a1454ab19dc8 service neutron] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:12.362465 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7596c1c2-d3e4-4eed-bbd5-a1454ab19dc8 service neutron] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:12.368170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac415d5-aab4-4570-9a3b-e556cfdab529 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2a6d7bd10435ba2c8e552d808b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:12.368820 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac415d5-aab4-4570-9a3b-e556cfdab529 service swift] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:12.369108 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac415d5-aab4-4570-9a3b-e556cfdab529 service swift] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:12.369384 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac415d5-aab4-4570-9a3b-e556cfdab529 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:12.378478 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac415d5-aab4-4570-9a3b-e556cfdab529 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:12.380092 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac415d5-aab4-4570-9a3b-e556cfdab529 service swift] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:12.388703 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1231/2434] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 14:19:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:19:17.019271 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98c74e-d449-4a7f-bb67-eafa773ddd69 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:17.019826 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98c74e-d449-4a7f-bb67-eafa773ddd69 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:17.019826 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98c74e-d449-4a7f-bb67-eafa773ddd69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:17.040371 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-df98c74e-d449-4a7f-bb67-eafa773ddd69 None None] MFA Rules not processed for user `2eb09df64dd74ddeb9fbbf44ae0a3592`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:19:17.041631 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-df98c74e-d449-4a7f-bb67-eafa773ddd69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:19:17.097782 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1204/2435] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:19:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:19:22.385621 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=30 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:19:27.091059 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=27 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:19:34.602707 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf4036-6613-4d71-9a67-691c007a9b06 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:34.603256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf4036-6613-4d71-9a67-691c007a9b06 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:34.603256 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf4036-6613-4d71-9a67-691c007a9b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:34.659424 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-18cf4036-6613-4d71-9a67-691c007a9b06 None None] MFA Rules not processed for user `5007a2f8e6244b0788f0a554cc572fe5`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:19:34.660718 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-18cf4036-6613-4d71-9a67-691c007a9b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:19:34.677630 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1232/2436] 10.222.0.48 () {58 vars in 967 bytes} [Mon Aug 1 14:19:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:19:34.690863 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df98c74e-d449-4a7f-bb67-eafa773ddd69 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:34.714431 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df98c74e-d449-4a7f-bb67-eafa773ddd69 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:34.719475 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720f6f1e-26bf-4bfa-b96c-9204dd276c74 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2a6d7bd10435ba2c8e552d808b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:34.720109 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f6f1e-26bf-4bfa-b96c-9204dd276c74 service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:34.720397 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f6f1e-26bf-4bfa-b96c-9204dd276c74 service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:34.720709 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f6f1e-26bf-4bfa-b96c-9204dd276c74 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:34.729679 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720f6f1e-26bf-4bfa-b96c-9204dd276c74 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:34.731283 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720f6f1e-26bf-4bfa-b96c-9204dd276c74 service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:34.740141 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1205/2437] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 14:19:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:19:44.674702 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=18 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:19:44.736605 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=10 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:19:49.225975 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf4036-6613-4d71-9a67-691c007a9b06 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:19:49.236046 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf4036-6613-4d71-9a67-691c007a9b06 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:19:49.239783 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c972817-248d-4641-9bd8-3a438c90dfb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:19:49.240444 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c972817-248d-4641-9bd8-3a438c90dfb5 service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:19:49.240772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c972817-248d-4641-9bd8-3a438c90dfb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:19:49.241071 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c972817-248d-4641-9bd8-3a438c90dfb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:19:49.242912 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-8c972817-248d-4641-9bd8-3a438c90dfb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:19:49.306506 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c972817-248d-4641-9bd8-3a438c90dfb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:19:49.308024 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c972817-248d-4641-9bd8-3a438c90dfb5 service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:19:49.318282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1233/2438] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:19:49 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:19:59.313769 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=30 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:08.831130 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720f6f1e-26bf-4bfa-b96c-9204dd276c74 service swift] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:08.841083 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720f6f1e-26bf-4bfa-b96c-9204dd276c74 service swift] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:08.844658 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291a60c-27b7-4141-91f2-2a15798b7bd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:08.845358 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291a60c-27b7-4141-91f2-2a15798b7bd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:08.845638 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291a60c-27b7-4141-91f2-2a15798b7bd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:08.845911 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291a60c-27b7-4141-91f2-2a15798b7bd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:08.847637 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e291a60c-27b7-4141-91f2-2a15798b7bd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:08.857604 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e291a60c-27b7-4141-91f2-2a15798b7bd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:08.859039 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e291a60c-27b7-4141-91f2-2a15798b7bd5 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:08.867352 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1206/2439] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:18.863647 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=6 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:43.987568 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c972817-248d-4641-9bd8-3a438c90dfb5 service neutron] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:43.989295 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-8c972817-248d-4641-9bd8-3a438c90dfb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:44.047264 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c972817-248d-4641-9bd8-3a438c90dfb5 service neutron] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:44.050879 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cfc80e-3a4c-488a-bfed-fc75ec5d9769 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:44.051497 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cfc80e-3a4c-488a-bfed-fc75ec5d9769 service cinder] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:44.051781 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cfc80e-3a4c-488a-bfed-fc75ec5d9769 service cinder] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:44.052065 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cfc80e-3a4c-488a-bfed-fc75ec5d9769 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:44.053913 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-21cfc80e-3a4c-488a-bfed-fc75ec5d9769 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:44.086708 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cfc80e-3a4c-488a-bfed-fc75ec5d9769 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:44.088148 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cfc80e-3a4c-488a-bfed-fc75ec5d9769 service cinder] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:44.106642 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1234/2440] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:54.104033 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=39 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:20:55.054636 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291a60c-27b7-4141-91f2-2a15798b7bd5 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:55.064679 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291a60c-27b7-4141-91f2-2a15798b7bd5 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:55.068531 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b9ef3-5bf1-442a-81ca-9863b70e6da9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:55.069157 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b9ef3-5bf1-442a-81ca-9863b70e6da9 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:55.069443 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b9ef3-5bf1-442a-81ca-9863b70e6da9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:55.069745 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b9ef3-5bf1-442a-81ca-9863b70e6da9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:55.072078 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d16b9ef3-5bf1-442a-81ca-9863b70e6da9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:55.168544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b9ef3-5bf1-442a-81ca-9863b70e6da9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:55.169978 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b9ef3-5bf1-442a-81ca-9863b70e6da9 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:55.191864 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1207/2441] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:20:56.094138 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cfc80e-3a4c-488a-bfed-fc75ec5d9769 service cinder] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:20:56.096455 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-21cfc80e-3a4c-488a-bfed-fc75ec5d9769 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:56.120044 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cfc80e-3a4c-488a-bfed-fc75ec5d9769 service cinder] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:20:56.127945 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c7b66-6260-44be-8597-1672b179b9fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:20:56.128614 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c7b66-6260-44be-8597-1672b179b9fa service nova] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:20:56.129418 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c7b66-6260-44be-8597-1672b179b9fa service nova] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:20:56.129711 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c7b66-6260-44be-8597-1672b179b9fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:20:56.131886 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-5c2c7b66-6260-44be-8597-1672b179b9fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:20:56.146648 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c7b66-6260-44be-8597-1672b179b9fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:20:56.148340 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c7b66-6260-44be-8597-1672b179b9fa service nova] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:20:56.161119 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1235/2442] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:20:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:01.985972 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b9ef3-5bf1-442a-81ca-9863b70e6da9 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:02.000202 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b9ef3-5bf1-442a-81ca-9863b70e6da9 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:02.010704 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377dc2e4-ed23-4950-a505-f3032cc2646a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:02.011744 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377dc2e4-ed23-4950-a505-f3032cc2646a service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:02.012462 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377dc2e4-ed23-4950-a505-f3032cc2646a service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:02.015488 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377dc2e4-ed23-4950-a505-f3032cc2646a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:02.021860 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-377dc2e4-ed23-4950-a505-f3032cc2646a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:02.155025 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377dc2e4-ed23-4950-a505-f3032cc2646a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:02.157858 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377dc2e4-ed23-4950-a505-f3032cc2646a service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:02.169550 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1208/2443] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:05.848418 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c7b66-6260-44be-8597-1672b179b9fa service nova] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:05.849888 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-5c2c7b66-6260-44be-8597-1672b179b9fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:05.863350 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c7b66-6260-44be-8597-1672b179b9fa service nova] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:05.868717 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a4e3f4-304d-4b61-bacb-63ef9b51943a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:05.869397 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4e3f4-304d-4b61-bacb-63ef9b51943a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:05.869685 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4e3f4-304d-4b61-bacb-63ef9b51943a admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:05.869967 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4e3f4-304d-4b61-bacb-63ef9b51943a admin admin] PATH_INFO: `/v3/users/62cc969defd64837a20e9a7dbda5b4fa` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:05.871155 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a4e3f4-304d-4b61-bacb-63ef9b51943a admin admin] RBAC: Authorizing `identity:delete_user(user_id=62cc969defd64837a20e9a7dbda5b4fa)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:05.873120 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a4e3f4-304d-4b61-bacb-63ef9b51943a admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:05.920170 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-e4a4e3f4-304d-4b61-bacb-63ef9b51943a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62cc969defd64837a20e9a7dbda5b4fa'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:05.920852 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-e4a4e3f4-304d-4b61-bacb-63ef9b51943a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62cc969defd64837a20e9a7dbda5b4fa'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:05.933188 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-e4a4e3f4-304d-4b61-bacb-63ef9b51943a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62cc969defd64837a20e9a7dbda5b4fa'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:05.940752 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-e4a4e3f4-304d-4b61-bacb-63ef9b51943a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62cc969defd64837a20e9a7dbda5b4fa'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:05.941398 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-e4a4e3f4-304d-4b61-bacb-63ef9b51943a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62cc969defd64837a20e9a7dbda5b4fa'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:05.954817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1236/2444] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:05 2022] DELETE /identity/v3/users/62cc969defd64837a20e9a7dbda5b4fa => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:06.141458 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377dc2e4-ed23-4950-a505-f3032cc2646a service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:06.154881 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377dc2e4-ed23-4950-a505-f3032cc2646a service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:06.212724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993fab7e-56f1-46b3-8db5-af6a787c526d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:06.213371 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993fab7e-56f1-46b3-8db5-af6a787c526d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:06.213657 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993fab7e-56f1-46b3-8db5-af6a787c526d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:06.213940 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993fab7e-56f1-46b3-8db5-af6a787c526d admin admin] PATH_INFO: `/v3/projects/6d4402cabc4b41e0aec0ccc40cdd17cc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:06.215086 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993fab7e-56f1-46b3-8db5-af6a787c526d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d4402cabc4b41e0aec0ccc40cdd17cc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:06.217025 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993fab7e-56f1-46b3-8db5-af6a787c526d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:06.223199 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:06.267710 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-993fab7e-56f1-46b3-8db5-af6a787c526d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d4402cabc4b41e0aec0ccc40cdd17cc', 'request_id': 'req-993fab7e-56f1-46b3-8db5-af6a787c526d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:06.277394 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-993fab7e-56f1-46b3-8db5-af6a787c526d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d4402cabc4b41e0aec0ccc40cdd17cc', 'request_id': 'req-993fab7e-56f1-46b3-8db5-af6a787c526d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:06.286275 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-993fab7e-56f1-46b3-8db5-af6a787c526d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d4402cabc4b41e0aec0ccc40cdd17cc', 'request_id': 'req-993fab7e-56f1-46b3-8db5-af6a787c526d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:06.297375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-993fab7e-56f1-46b3-8db5-af6a787c526d admin admin] The token cache is being invalidate because project 6d4402cabc4b41e0aec0ccc40cdd17cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:06.297673 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-993fab7e-56f1-46b3-8db5-af6a787c526d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:06.298388 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-993fab7e-56f1-46b3-8db5-af6a787c526d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:06.300169 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1209/2445] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:06 2022] DELETE /identity/v3/projects/6d4402cabc4b41e0aec0ccc40cdd17cc => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:06.305126 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e9858-fe82-4811-85a1-c7eb07b80dbb None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:06.305442 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e9858-fe82-4811-85a1-c7eb07b80dbb None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:06.305723 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e9858-fe82-4811-85a1-c7eb07b80dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:06.328979 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-dc5e9858-fe82-4811-85a1-c7eb07b80dbb None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:06.330140 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-dc5e9858-fe82-4811-85a1-c7eb07b80dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:06.393092 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1237/2446] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:06.402060 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dab044-d2cf-46ba-9255-3a274e5fedc9 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:06.402341 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dab044-d2cf-46ba-9255-3a274e5fedc9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:06.402624 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dab044-d2cf-46ba-9255-3a274e5fedc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:06.432673 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-18dab044-d2cf-46ba-9255-3a274e5fedc9 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:06.432673 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-18dab044-d2cf-46ba-9255-3a274e5fedc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:06.435539 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1210/2447] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:06.439770 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e9858-fe82-4811-85a1-c7eb07b80dbb None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:06.450659 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e9858-fe82-4811-85a1-c7eb07b80dbb None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:06.456237 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee870926-43dd-4bd4-afbf-c6433b2427e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:06.456914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee870926-43dd-4bd4-afbf-c6433b2427e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:06.457196 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee870926-43dd-4bd4-afbf-c6433b2427e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:06.457473 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee870926-43dd-4bd4-afbf-c6433b2427e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:06.458180 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee870926-43dd-4bd4-afbf-c6433b2427e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:21:06.458484 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee870926-43dd-4bd4-afbf-c6433b2427e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:06.460121 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee870926-43dd-4bd4-afbf-c6433b2427e9 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:06.473172 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1238/2448] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:21:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:21:06.477498 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dab044-d2cf-46ba-9255-3a274e5fedc9 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:06.487011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dab044-d2cf-46ba-9255-3a274e5fedc9 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:06.492259 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8636f-7424-4851-be9e-9d05e305d528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:06.492958 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8636f-7424-4851-be9e-9d05e305d528 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:06.493246 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8636f-7424-4851-be9e-9d05e305d528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:06.493530 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8636f-7424-4851-be9e-9d05e305d528 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:06.494230 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8636f-7424-4851-be9e-9d05e305d528 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:06.496123 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8636f-7424-4851-be9e-9d05e305d528 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:06.532849 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1211/2449] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:06 2022] POST /identity/v3/projects => generated 355 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:21:06.537086 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee870926-43dd-4bd4-afbf-c6433b2427e9 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:06.547108 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee870926-43dd-4bd4-afbf-c6433b2427e9 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:06.593178 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:06.607957 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca7e2dd-f249-4b50-8bab-e2265c0be83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:06.608619 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7e2dd-f249-4b50-8bab-e2265c0be83b admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:06.608904 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7e2dd-f249-4b50-8bab-e2265c0be83b admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:06.609198 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7e2dd-f249-4b50-8bab-e2265c0be83b admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:06.609870 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca7e2dd-f249-4b50-8bab-e2265c0be83b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:06.612698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca7e2dd-f249-4b50-8bab-e2265c0be83b admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:06.634788 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1239/2450] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:06 2022] POST /identity/v3/users => generated 278 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:06.639061 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8636f-7424-4851-be9e-9d05e305d528 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:06.650949 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8636f-7424-4851-be9e-9d05e305d528 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:06.655649 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6decb9d-e7c2-4fcb-b884-afff942bbfd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:06.656270 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6decb9d-e7c2-4fcb-b884-afff942bbfd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:06.656591 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6decb9d-e7c2-4fcb-b884-afff942bbfd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:06.656877 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6decb9d-e7c2-4fcb-b884-afff942bbfd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:06.657576 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6decb9d-e7c2-4fcb-b884-afff942bbfd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:06.659161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6decb9d-e7c2-4fcb-b884-afff942bbfd6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:06.669083 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1212/2451] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:06.673079 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca7e2dd-f249-4b50-8bab-e2265c0be83b admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:06.684024 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca7e2dd-f249-4b50-8bab-e2265c0be83b admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:06.689688 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ce30c-02cf-40f3-8068-362e37e785cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:06.690330 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ce30c-02cf-40f3-8068-362e37e785cf admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:06.690579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ce30c-02cf-40f3-8068-362e37e785cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:06.690885 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ce30c-02cf-40f3-8068-362e37e785cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:06.691589 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664ce30c-02cf-40f3-8068-362e37e785cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:06.693281 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664ce30c-02cf-40f3-8068-362e37e785cf admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:06.704357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1240/2452] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:06.708699 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6decb9d-e7c2-4fcb-b884-afff942bbfd6 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:06.719370 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6decb9d-e7c2-4fcb-b884-afff942bbfd6 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:06.724126 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c156dced-9d5a-43ed-a637-616aea7fab26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:06.724820 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156dced-9d5a-43ed-a637-616aea7fab26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:06.725105 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156dced-9d5a-43ed-a637-616aea7fab26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:06.725387 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156dced-9d5a-43ed-a637-616aea7fab26 admin admin] PATH_INFO: `/v3/projects/efe741d706974bb0b475d7450a87fe26/users/50ef2b0200604e9995c89f757521825a/roles/926b9f36c24847c582c84af904250abc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:06.741463 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c156dced-9d5a-43ed-a637-616aea7fab26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efe741d706974bb0b475d7450a87fe26, user_id=50ef2b0200604e9995c89f757521825a, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:06.744655 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c156dced-9d5a-43ed-a637-616aea7fab26 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:06.758510 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1213/2453] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:06 2022] PUT /identity/v3/projects/efe741d706974bb0b475d7450a87fe26/users/50ef2b0200604e9995c89f757521825a/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:06.770964 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439403b3-b2de-4a4a-83c2-fae418abb3da None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:06.771432 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439403b3-b2de-4a4a-83c2-fae418abb3da None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:06.771909 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439403b3-b2de-4a4a-83c2-fae418abb3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:06.806966 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-439403b3-b2de-4a4a-83c2-fae418abb3da None None] MFA Rules not processed for user `50ef2b0200604e9995c89f757521825a`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:06.808196 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-439403b3-b2de-4a4a-83c2-fae418abb3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:06.894192 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1241/2454] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:06 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:06.899210 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c156dced-9d5a-43ed-a637-616aea7fab26 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:06.961230 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c156dced-9d5a-43ed-a637-616aea7fab26 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:06.965907 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51358b7b-7f24-491b-8b06-0e813614c735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:06.966576 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51358b7b-7f24-491b-8b06-0e813614c735 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:06.966863 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51358b7b-7f24-491b-8b06-0e813614c735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:06.967168 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51358b7b-7f24-491b-8b06-0e813614c735 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:06.967857 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51358b7b-7f24-491b-8b06-0e813614c735 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:06.969931 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51358b7b-7f24-491b-8b06-0e813614c735 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:06.995941 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1214/2455] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:06 2022] POST /identity/v3/projects => generated 357 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:07.000222 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439403b3-b2de-4a4a-83c2-fae418abb3da None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:07.011439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439403b3-b2de-4a4a-83c2-fae418abb3da None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:07.067702 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1b3fbf-81d3-4a57-af7b-d2611d196f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:07.068376 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b3fbf-81d3-4a57-af7b-d2611d196f95 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:07.068679 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b3fbf-81d3-4a57-af7b-d2611d196f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:07.068961 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b3fbf-81d3-4a57-af7b-d2611d196f95 admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:07.070127 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1b3fbf-81d3-4a57-af7b-d2611d196f95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:07.072078 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1b3fbf-81d3-4a57-af7b-d2611d196f95 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:07.093002 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1242/2456] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:06 2022] POST /identity/v3/users => generated 285 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:07.097130 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51358b7b-7f24-491b-8b06-0e813614c735 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:07.107259 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51358b7b-7f24-491b-8b06-0e813614c735 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:07.111955 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae12f5f-162b-4742-bf12-d54a88f17741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:07.112602 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae12f5f-162b-4742-bf12-d54a88f17741 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:07.112891 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae12f5f-162b-4742-bf12-d54a88f17741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:07.113184 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae12f5f-162b-4742-bf12-d54a88f17741 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:07.113869 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae12f5f-162b-4742-bf12-d54a88f17741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:07.115569 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae12f5f-162b-4742-bf12-d54a88f17741 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:07.116956 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:07.129950 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1215/2457] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:07.134372 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1b3fbf-81d3-4a57-af7b-d2611d196f95 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:07.144441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1b3fbf-81d3-4a57-af7b-d2611d196f95 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:07.149811 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31227e6d-b188-4bb6-a48a-90628f461564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:07.150481 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31227e6d-b188-4bb6-a48a-90628f461564 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:07.150768 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31227e6d-b188-4bb6-a48a-90628f461564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:07.151048 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31227e6d-b188-4bb6-a48a-90628f461564 admin admin] PATH_INFO: `/v3/projects/ef9a0bb54d6249468a6788594e930b92/users/9bf3ebe8ea3943638f7db3ef1966bb7e/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:07.166563 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31227e6d-b188-4bb6-a48a-90628f461564 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef9a0bb54d6249468a6788594e930b92, user_id=9bf3ebe8ea3943638f7db3ef1966bb7e, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:07.169274 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31227e6d-b188-4bb6-a48a-90628f461564 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:07.182598 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1243/2458] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:07 2022] PUT /identity/v3/projects/ef9a0bb54d6249468a6788594e930b92/users/9bf3ebe8ea3943638f7db3ef1966bb7e/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:07.186902 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae12f5f-162b-4742-bf12-d54a88f17741 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:07.205758 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae12f5f-162b-4742-bf12-d54a88f17741 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:07.277603 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba495fd-d25c-4b35-802a-6d62dbf79243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:07.278325 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba495fd-d25c-4b35-802a-6d62dbf79243 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:07.278613 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba495fd-d25c-4b35-802a-6d62dbf79243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:07.278895 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba495fd-d25c-4b35-802a-6d62dbf79243 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:07.279597 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba495fd-d25c-4b35-802a-6d62dbf79243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:07.281225 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba495fd-d25c-4b35-802a-6d62dbf79243 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:07.294009 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1216/2459] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:07.298306 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31227e6d-b188-4bb6-a48a-90628f461564 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:07.310778 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31227e6d-b188-4bb6-a48a-90628f461564 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:07.315933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77edaf-60c4-46fe-b59d-101df14f0ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:07.316648 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77edaf-60c4-46fe-b59d-101df14f0ecc admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:07.316936 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77edaf-60c4-46fe-b59d-101df14f0ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:07.317220 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77edaf-60c4-46fe-b59d-101df14f0ecc admin admin] PATH_INFO: `/v3/projects/ef9a0bb54d6249468a6788594e930b92/users/9bf3ebe8ea3943638f7db3ef1966bb7e/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:07.319195 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f77edaf-60c4-46fe-b59d-101df14f0ecc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef9a0bb54d6249468a6788594e930b92, user_id=9bf3ebe8ea3943638f7db3ef1966bb7e, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:07.323099 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f77edaf-60c4-46fe-b59d-101df14f0ecc admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:07.339095 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1244/2460] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:07 2022] PUT /identity/v3/projects/ef9a0bb54d6249468a6788594e930b92/users/9bf3ebe8ea3943638f7db3ef1966bb7e/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:07.349344 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef955b91-505b-4356-8910-4160f0582df9 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:07.349637 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef955b91-505b-4356-8910-4160f0582df9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:07.349921 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef955b91-505b-4356-8910-4160f0582df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:07.376981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ef955b91-505b-4356-8910-4160f0582df9 None None] MFA Rules not processed for user `9bf3ebe8ea3943638f7db3ef1966bb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:07.378196 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ef955b91-505b-4356-8910-4160f0582df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:07.445488 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1217/2461] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:07 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:07.456691 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77edaf-60c4-46fe-b59d-101df14f0ecc admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:07.458585 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-5f77edaf-60c4-46fe-b59d-101df14f0ecc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:07.469869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:07.521653 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77edaf-60c4-46fe-b59d-101df14f0ecc admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:07.525423 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57582931-a195-4f72-b738-73f92d7b1923 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:07.526030 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57582931-a195-4f72-b738-73f92d7b1923 service cinder] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:07.526316 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57582931-a195-4f72-b738-73f92d7b1923 service cinder] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:07.526595 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57582931-a195-4f72-b738-73f92d7b1923 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:07.536903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57582931-a195-4f72-b738-73f92d7b1923 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:07.538282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57582931-a195-4f72-b738-73f92d7b1923 service cinder] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:07.548431 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1245/2462] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:07 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:07.627555 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef955b91-505b-4356-8910-4160f0582df9 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:07.629079 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ef955b91-505b-4356-8910-4160f0582df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:07.641580 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef955b91-505b-4356-8910-4160f0582df9 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:07.646119 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a642a4f-4e2e-4ce2-9553-76c0bdf1b83c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:07.646776 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a642a4f-4e2e-4ce2-9553-76c0bdf1b83c service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:07.647067 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a642a4f-4e2e-4ce2-9553-76c0bdf1b83c service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:07.647347 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a642a4f-4e2e-4ce2-9553-76c0bdf1b83c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:07.727560 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a642a4f-4e2e-4ce2-9553-76c0bdf1b83c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:07.729052 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a642a4f-4e2e-4ce2-9553-76c0bdf1b83c service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:07.737919 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1218/2463] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:07 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:08.446785 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57582931-a195-4f72-b738-73f92d7b1923 service cinder] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:08.448200 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-57582931-a195-4f72-b738-73f92d7b1923 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:08.504368 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57582931-a195-4f72-b738-73f92d7b1923 service cinder] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:08.507986 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc7618-4793-4e22-a86c-221dc34d4598 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:08.508639 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc7618-4793-4e22-a86c-221dc34d4598 service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:08.508925 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc7618-4793-4e22-a86c-221dc34d4598 service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:08.509203 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc7618-4793-4e22-a86c-221dc34d4598 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:08.510954 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-15cc7618-4793-4e22-a86c-221dc34d4598 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:08.573033 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cc7618-4793-4e22-a86c-221dc34d4598 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:08.574459 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cc7618-4793-4e22-a86c-221dc34d4598 service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:08.583915 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1246/2464] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:08.816125 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a642a4f-4e2e-4ce2-9553-76c0bdf1b83c service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:08.818082 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4a642a4f-4e2e-4ce2-9553-76c0bdf1b83c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:08.830278 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a642a4f-4e2e-4ce2-9553-76c0bdf1b83c service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:08.833852 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573a668-385b-4481-b40b-60d1ba62e1f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:08.834467 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573a668-385b-4481-b40b-60d1ba62e1f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:08.834756 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573a668-385b-4481-b40b-60d1ba62e1f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:08.835039 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573a668-385b-4481-b40b-60d1ba62e1f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:08.837313 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2573a668-385b-4481-b40b-60d1ba62e1f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:08.849124 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573a668-385b-4481-b40b-60d1ba62e1f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:08.850562 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573a668-385b-4481-b40b-60d1ba62e1f4 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:08.861196 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1219/2465] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:12.328937 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc7618-4793-4e22-a86c-221dc34d4598 service neutron] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:12.337966 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-15cc7618-4793-4e22-a86c-221dc34d4598 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:12.373925 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc7618-4793-4e22-a86c-221dc34d4598 service neutron] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:12.389734 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0768fd-a48c-4d6e-8815-6367ea13440f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:12.390373 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0768fd-a48c-4d6e-8815-6367ea13440f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:12.390661 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0768fd-a48c-4d6e-8815-6367ea13440f admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:12.390939 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0768fd-a48c-4d6e-8815-6367ea13440f admin admin] PATH_INFO: `/v3/users/ebe4b3455a3d4139a1267da97bd83662` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:12.413922 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0768fd-a48c-4d6e-8815-6367ea13440f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebe4b3455a3d4139a1267da97bd83662)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:12.416534 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0768fd-a48c-4d6e-8815-6367ea13440f admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:12.496959 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:12.500003 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-8e0768fd-a48c-4d6e-8815-6367ea13440f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebe4b3455a3d4139a1267da97bd83662'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:12.501449 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-8e0768fd-a48c-4d6e-8815-6367ea13440f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebe4b3455a3d4139a1267da97bd83662'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:12.523853 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-8e0768fd-a48c-4d6e-8815-6367ea13440f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebe4b3455a3d4139a1267da97bd83662'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:12.539546 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-8e0768fd-a48c-4d6e-8815-6367ea13440f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebe4b3455a3d4139a1267da97bd83662'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:12.540198 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-8e0768fd-a48c-4d6e-8815-6367ea13440f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebe4b3455a3d4139a1267da97bd83662'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:12.562392 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1247/2466] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:12 2022] DELETE /identity/v3/users/ebe4b3455a3d4139a1267da97bd83662 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:12.625596 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573a668-385b-4481-b40b-60d1ba62e1f4 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:12.627962 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2573a668-385b-4481-b40b-60d1ba62e1f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:12.634783 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:12.697568 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0768fd-a48c-4d6e-8815-6367ea13440f admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:12.699250 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-8e0768fd-a48c-4d6e-8815-6367ea13440f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:12.728603 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573a668-385b-4481-b40b-60d1ba62e1f4 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:12.810264 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0768fd-a48c-4d6e-8815-6367ea13440f admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:12.820828 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a840b50-70d1-4c77-bb1c-507105402f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:12.820828 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a840b50-70d1-4c77-bb1c-507105402f41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:12.820828 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a840b50-70d1-4c77-bb1c-507105402f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:12.820828 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a840b50-70d1-4c77-bb1c-507105402f41 admin admin] PATH_INFO: `/v3/users/ca3cd1c2aec94b43aff0034b812a069d` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:12.821518 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a840b50-70d1-4c77-bb1c-507105402f41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca3cd1c2aec94b43aff0034b812a069d)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:12.823764 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a840b50-70d1-4c77-bb1c-507105402f41 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:12.826045 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f932e1-c69a-4a71-bafc-13bbc7ca06c8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e294fe796a1a4ac89041a532856ad627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:12.826230 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f932e1-c69a-4a71-bafc-13bbc7ca06c8 service placement] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:12.826739 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f932e1-c69a-4a71-bafc-13bbc7ca06c8 service placement] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:12.827088 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f932e1-c69a-4a71-bafc-13bbc7ca06c8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:12.830845 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f6f932e1-c69a-4a71-bafc-13bbc7ca06c8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:12.844088 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f932e1-c69a-4a71-bafc-13bbc7ca06c8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:12.846003 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f932e1-c69a-4a71-bafc-13bbc7ca06c8 service placement] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:12.858726 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1220/2467] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:12.859642 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f932e1-c69a-4a71-bafc-13bbc7ca06c8 service placement] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:12.861148 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f6f932e1-c69a-4a71-bafc-13bbc7ca06c8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:12.875769 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f932e1-c69a-4a71-bafc-13bbc7ca06c8 service placement] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:12.882993 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3abcca-5979-43cf-8a33-29a239b64e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:12.883880 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3abcca-5979-43cf-8a33-29a239b64e04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:12.884168 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3abcca-5979-43cf-8a33-29a239b64e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:12.884573 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3abcca-5979-43cf-8a33-29a239b64e04 admin admin] PATH_INFO: `/v3/projects/c3d2b7b0783c4cc7bd0ce59a13f70efa` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:12.906380 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-4a840b50-70d1-4c77-bb1c-507105402f41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca3cd1c2aec94b43aff0034b812a069d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:12.907407 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-4a840b50-70d1-4c77-bb1c-507105402f41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca3cd1c2aec94b43aff0034b812a069d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:12.912720 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3abcca-5979-43cf-8a33-29a239b64e04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3d2b7b0783c4cc7bd0ce59a13f70efa)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:12.916100 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3abcca-5979-43cf-8a33-29a239b64e04 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:12.924240 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-4a840b50-70d1-4c77-bb1c-507105402f41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca3cd1c2aec94b43aff0034b812a069d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:12.933287 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-4a840b50-70d1-4c77-bb1c-507105402f41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca3cd1c2aec94b43aff0034b812a069d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:12.934009 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-4a840b50-70d1-4c77-bb1c-507105402f41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca3cd1c2aec94b43aff0034b812a069d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:12.949057 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1248/2468] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:12 2022] DELETE /identity/v3/users/ca3cd1c2aec94b43aff0034b812a069d => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:12.974236 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7d3abcca-5979-43cf-8a33-29a239b64e04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3d2b7b0783c4cc7bd0ce59a13f70efa', 'request_id': 'req-7d3abcca-5979-43cf-8a33-29a239b64e04'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:12.983305 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7d3abcca-5979-43cf-8a33-29a239b64e04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3d2b7b0783c4cc7bd0ce59a13f70efa', 'request_id': 'req-7d3abcca-5979-43cf-8a33-29a239b64e04'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:12.990670 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7d3abcca-5979-43cf-8a33-29a239b64e04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3d2b7b0783c4cc7bd0ce59a13f70efa', 'request_id': 'req-7d3abcca-5979-43cf-8a33-29a239b64e04'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:13.002411 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7d3abcca-5979-43cf-8a33-29a239b64e04 admin admin] The token cache is being invalidate because project c3d2b7b0783c4cc7bd0ce59a13f70efa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:13.003026 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7d3abcca-5979-43cf-8a33-29a239b64e04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:13.004628 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7d3abcca-5979-43cf-8a33-29a239b64e04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:13.006502 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1221/2469] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:12 2022] DELETE /identity/v3/projects/c3d2b7b0783c4cc7bd0ce59a13f70efa => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:13.011553 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072ef37-3a9d-409a-8a9a-2ad2ffb21346 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:13.011868 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072ef37-3a9d-409a-8a9a-2ad2ffb21346 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:13.012153 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072ef37-3a9d-409a-8a9a-2ad2ffb21346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:13.033194 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-6072ef37-3a9d-409a-8a9a-2ad2ffb21346 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:13.034408 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-6072ef37-3a9d-409a-8a9a-2ad2ffb21346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:13.072195 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:13.100067 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1249/2470] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:13.108438 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875c26d-23f6-417b-961f-7c9f4ec0b6f3 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:13.108782 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875c26d-23f6-417b-961f-7c9f4ec0b6f3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:13.109061 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875c26d-23f6-417b-961f-7c9f4ec0b6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:13.130234 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8875c26d-23f6-417b-961f-7c9f4ec0b6f3 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:13.131455 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8875c26d-23f6-417b-961f-7c9f4ec0b6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:13.134416 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6072ef37-3a9d-409a-8a9a-2ad2ffb21346 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:13.136351 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-6072ef37-3a9d-409a-8a9a-2ad2ffb21346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:13.141424 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1222/2471] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:13.145603 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875c26d-23f6-417b-961f-7c9f4ec0b6f3 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:13.149624 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6072ef37-3a9d-409a-8a9a-2ad2ffb21346 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:13.154845 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf9729-2ce5-4113-97e9-72ba7db437c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:13.155502 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf9729-2ce5-4113-97e9-72ba7db437c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:13.155786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf9729-2ce5-4113-97e9-72ba7db437c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:13.156064 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf9729-2ce5-4113-97e9-72ba7db437c2 admin admin] PATH_INFO: `/v3/projects/a3203f080a05422e83090a1f0a31d1a9` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:13.157259 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcf9729-2ce5-4113-97e9-72ba7db437c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3203f080a05422e83090a1f0a31d1a9)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:13.159163 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcf9729-2ce5-4113-97e9-72ba7db437c2 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:13.160347 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:13.168053 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875c26d-23f6-417b-961f-7c9f4ec0b6f3 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:13.175555 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5addc7-0ed4-4593-a44a-e2b8f85b23ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:13.176176 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5addc7-0ed4-4593-a44a-e2b8f85b23ef admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:13.176452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5addc7-0ed4-4593-a44a-e2b8f85b23ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:13.176754 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5addc7-0ed4-4593-a44a-e2b8f85b23ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:13.177457 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5addc7-0ed4-4593-a44a-e2b8f85b23ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:21:13.177760 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5addc7-0ed4-4593-a44a-e2b8f85b23ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:13.179890 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5addc7-0ed4-4593-a44a-e2b8f85b23ef admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:13.192485 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1223/2472] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:21:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:21:13.196855 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5addc7-0ed4-4593-a44a-e2b8f85b23ef admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:13.207020 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5addc7-0ed4-4593-a44a-e2b8f85b23ef admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:13.210515 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-efcf9729-2ce5-4113-97e9-72ba7db437c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3203f080a05422e83090a1f0a31d1a9', 'request_id': 'req-efcf9729-2ce5-4113-97e9-72ba7db437c2'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:13.217554 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-efcf9729-2ce5-4113-97e9-72ba7db437c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3203f080a05422e83090a1f0a31d1a9', 'request_id': 'req-efcf9729-2ce5-4113-97e9-72ba7db437c2'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:13.223162 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-efcf9729-2ce5-4113-97e9-72ba7db437c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3203f080a05422e83090a1f0a31d1a9', 'request_id': 'req-efcf9729-2ce5-4113-97e9-72ba7db437c2'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:13.236989 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-efcf9729-2ce5-4113-97e9-72ba7db437c2 admin admin] The token cache is being invalidate because project a3203f080a05422e83090a1f0a31d1a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:13.237310 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-efcf9729-2ce5-4113-97e9-72ba7db437c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:13.237976 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-efcf9729-2ce5-4113-97e9-72ba7db437c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:13.239533 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1250/2473] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:13 2022] DELETE /identity/v3/projects/a3203f080a05422e83090a1f0a31d1a9 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:13.244829 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e53aa1-4e58-424f-b8ba-e0d57b0ef535 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:13.245124 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e53aa1-4e58-424f-b8ba-e0d57b0ef535 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:13.245441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e53aa1-4e58-424f-b8ba-e0d57b0ef535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:13.265392 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-d5e53aa1-4e58-424f-b8ba-e0d57b0ef535 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:13.266099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686bb6d-4895-47a4-bc0f-8c44ee26807d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:13.266784 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686bb6d-4895-47a4-bc0f-8c44ee26807d admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:13.266932 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-d5e53aa1-4e58-424f-b8ba-e0d57b0ef535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:13.267096 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686bb6d-4895-47a4-bc0f-8c44ee26807d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:13.267340 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686bb6d-4895-47a4-bc0f-8c44ee26807d admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:13.268003 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4686bb6d-4895-47a4-bc0f-8c44ee26807d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:13.270025 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4686bb6d-4895-47a4-bc0f-8c44ee26807d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:13.275347 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1251/2474] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:21:13.283296 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498e254-c99c-475f-9491-42e2131cd691 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:13.283600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498e254-c99c-475f-9491-42e2131cd691 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:13.283876 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498e254-c99c-475f-9491-42e2131cd691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:13.299211 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1224/2475] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:13 2022] POST /identity/v3/projects => generated 405 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:13.303324 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686bb6d-4895-47a4-bc0f-8c44ee26807d admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:13.304356 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-9498e254-c99c-475f-9491-42e2131cd691 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:13.305110 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4686bb6d-4895-47a4-bc0f-8c44ee26807d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:13.305716 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-9498e254-c99c-475f-9491-42e2131cd691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:13.364696 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1252/2476] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:13.369064 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9498e254-c99c-475f-9491-42e2131cd691 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:13.372444 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686bb6d-4895-47a4-bc0f-8c44ee26807d admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:13.377034 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d02739-45a7-4c2c-a07e-6b36725205fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:13.377649 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d02739-45a7-4c2c-a07e-6b36725205fb admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:13.377850 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9498e254-c99c-475f-9491-42e2131cd691 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:13.377972 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d02739-45a7-4c2c-a07e-6b36725205fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:13.378191 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d02739-45a7-4c2c-a07e-6b36725205fb admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:13.378847 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d02739-45a7-4c2c-a07e-6b36725205fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:13.380781 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d02739-45a7-4c2c-a07e-6b36725205fb admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:13.382622 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f5d8d-e5b3-4d45-90f1-663c49ff4821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:13.383390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f5d8d-e5b3-4d45-90f1-663c49ff4821 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:13.383737 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f5d8d-e5b3-4d45-90f1-663c49ff4821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:13.384098 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f5d8d-e5b3-4d45-90f1-663c49ff4821 admin admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:13.384832 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f5d8d-e5b3-4d45-90f1-663c49ff4821 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:21:13.385193 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f5d8d-e5b3-4d45-90f1-663c49ff4821 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:13.386835 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f5d8d-e5b3-4d45-90f1-663c49ff4821 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:13.399147 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1253/2477] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:21:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:21:13.403015 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1225/2478] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:13 2022] POST /identity/v3/users => generated 303 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:21:13.403591 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f5d8d-e5b3-4d45-90f1-663c49ff4821 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:13.406690 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d02739-45a7-4c2c-a07e-6b36725205fb admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:13.414265 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f5d8d-e5b3-4d45-90f1-663c49ff4821 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:13.416850 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d02739-45a7-4c2c-a07e-6b36725205fb admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:13.419057 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591381b9-55a5-491c-a465-7a1f6e72e58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:13.419690 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591381b9-55a5-491c-a465-7a1f6e72e58e admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:13.419976 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591381b9-55a5-491c-a465-7a1f6e72e58e admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:13.420284 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591381b9-55a5-491c-a465-7a1f6e72e58e admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:13.421013 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591381b9-55a5-491c-a465-7a1f6e72e58e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:13.421907 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b2d351-82e4-4d85-95e8-a1838f3acd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:13.422652 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b2d351-82e4-4d85-95e8-a1838f3acd2b admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:13.422907 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591381b9-55a5-491c-a465-7a1f6e72e58e admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:13.423064 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b2d351-82e4-4d85-95e8-a1838f3acd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:13.423418 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b2d351-82e4-4d85-95e8-a1838f3acd2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:13.424197 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b2d351-82e4-4d85-95e8-a1838f3acd2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:13.425911 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b2d351-82e4-4d85-95e8-a1838f3acd2b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:13.439691 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1226/2479] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:21:13.440853 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b2d351-82e4-4d85-95e8-a1838f3acd2b admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:13.453252 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1254/2480] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:13 2022] POST /identity/v3/projects => generated 403 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:21:13.453418 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b2d351-82e4-4d85-95e8-a1838f3acd2b admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:13.454523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591381b9-55a5-491c-a465-7a1f6e72e58e admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:13.483447 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:13.501935 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:13.516222 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d3afa9-94bd-428d-b1c9-130dd6016f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:13.516850 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3afa9-94bd-428d-b1c9-130dd6016f6c admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:13.517205 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3afa9-94bd-428d-b1c9-130dd6016f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:13.517448 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3afa9-94bd-428d-b1c9-130dd6016f6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:13.518101 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d3afa9-94bd-428d-b1c9-130dd6016f6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:13.519724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d3afa9-94bd-428d-b1c9-130dd6016f6c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:13.531988 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1227/2481] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:13.533127 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591381b9-55a5-491c-a465-7a1f6e72e58e admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:13.536268 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d3afa9-94bd-428d-b1c9-130dd6016f6c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:13.539153 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c51fd0-c1f6-4d2e-8e83-5d6e604a776b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:13.539781 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c51fd0-c1f6-4d2e-8e83-5d6e604a776b admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:13.540066 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c51fd0-c1f6-4d2e-8e83-5d6e604a776b admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:13.540344 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c51fd0-c1f6-4d2e-8e83-5d6e604a776b admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:13.541042 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c51fd0-c1f6-4d2e-8e83-5d6e604a776b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:13.542939 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c51fd0-c1f6-4d2e-8e83-5d6e604a776b admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:13.546853 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d3afa9-94bd-428d-b1c9-130dd6016f6c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:13.552650 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e8e6e-5a54-4760-b756-c1a302c2309d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:13.554122 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e8e6e-5a54-4760-b756-c1a302c2309d admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:13.554418 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e8e6e-5a54-4760-b756-c1a302c2309d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:13.554839 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e8e6e-5a54-4760-b756-c1a302c2309d admin admin] PATH_INFO: `/v3/projects/bbad1c32ba0b497bb7e0c39645d76ff3/users/41561005400b4b07aee87c425c15d5e8/roles/926b9f36c24847c582c84af904250abc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:13.567601 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1255/2482] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:13 2022] POST /identity/v3/users => generated 302 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:21:13.571700 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e8e6e-5a54-4760-b756-c1a302c2309d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbad1c32ba0b497bb7e0c39645d76ff3, user_id=41561005400b4b07aee87c425c15d5e8, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:13.573258 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c51fd0-c1f6-4d2e-8e83-5d6e604a776b admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:13.574860 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e8e6e-5a54-4760-b756-c1a302c2309d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:13.583515 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c51fd0-c1f6-4d2e-8e83-5d6e604a776b admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:13.590669 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1228/2483] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:13 2022] PUT /identity/v3/projects/bbad1c32ba0b497bb7e0c39645d76ff3/users/41561005400b4b07aee87c425c15d5e8/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:13.604027 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8099d-f96e-4398-b697-d1ca4e380791 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:13.604365 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8099d-f96e-4398-b697-d1ca4e380791 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:13.604610 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8099d-f96e-4398-b697-d1ca4e380791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:13.635333 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ecb8099d-f96e-4398-b697-d1ca4e380791 None None] MFA Rules not processed for user `41561005400b4b07aee87c425c15d5e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:13.637267 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ecb8099d-f96e-4398-b697-d1ca4e380791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:13.655678 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14db0ed5-c6fd-483e-a534-8bb1146f8936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:13.656309 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db0ed5-c6fd-483e-a534-8bb1146f8936 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:13.656632 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db0ed5-c6fd-483e-a534-8bb1146f8936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:13.656911 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db0ed5-c6fd-483e-a534-8bb1146f8936 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:13.657619 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14db0ed5-c6fd-483e-a534-8bb1146f8936 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:13.659228 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14db0ed5-c6fd-483e-a534-8bb1146f8936 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:13.671913 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1256/2484] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:13.675965 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14db0ed5-c6fd-483e-a534-8bb1146f8936 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:13.687539 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14db0ed5-c6fd-483e-a534-8bb1146f8936 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:13.693207 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb65f72-45c1-4be9-97d0-a918fe4e0846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:13.693804 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb65f72-45c1-4be9-97d0-a918fe4e0846 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:13.694639 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb65f72-45c1-4be9-97d0-a918fe4e0846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:13.694936 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb65f72-45c1-4be9-97d0-a918fe4e0846 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:13.695681 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb65f72-45c1-4be9-97d0-a918fe4e0846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:13.697429 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb65f72-45c1-4be9-97d0-a918fe4e0846 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:13.707343 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1229/2485] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:13 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:13.709088 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1257/2486] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:21:13.712566 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8099d-f96e-4398-b697-d1ca4e380791 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:13.713519 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb65f72-45c1-4be9-97d0-a918fe4e0846 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:13.723473 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8099d-f96e-4398-b697-d1ca4e380791 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:13.725559 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb65f72-45c1-4be9-97d0-a918fe4e0846 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:13.728461 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795480fc-8af7-455b-ba04-bcffa76f928d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:13.729107 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795480fc-8af7-455b-ba04-bcffa76f928d admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:13.729416 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795480fc-8af7-455b-ba04-bcffa76f928d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:13.729696 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795480fc-8af7-455b-ba04-bcffa76f928d admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:13.730356 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795480fc-8af7-455b-ba04-bcffa76f928d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:13.730481 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b3309f-bcdf-4bfb-9a8d-cb2e29ab974f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:13.731059 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3309f-bcdf-4bfb-9a8d-cb2e29ab974f admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:13.731339 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3309f-bcdf-4bfb-9a8d-cb2e29ab974f admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:13.731614 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3309f-bcdf-4bfb-9a8d-cb2e29ab974f admin admin] PATH_INFO: `/v3/projects/017be9a7280a4c189ee057f05a0a79d7/users/6c39041deecc4da0ac3648636b9eff86/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:13.732391 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795480fc-8af7-455b-ba04-bcffa76f928d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:13.746396 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b3309f-bcdf-4bfb-9a8d-cb2e29ab974f admin admin] RBAC: Authorizing `identity:create_grant(project_id=017be9a7280a4c189ee057f05a0a79d7, user_id=6c39041deecc4da0ac3648636b9eff86, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:13.749083 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b3309f-bcdf-4bfb-9a8d-cb2e29ab974f admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:13.763549 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1230/2487] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:13 2022] POST /identity/v3/projects => generated 401 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:21:13.763744 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1258/2488] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:13 2022] PUT /identity/v3/projects/017be9a7280a4c189ee057f05a0a79d7/users/6c39041deecc4da0ac3648636b9eff86/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:13.764502 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795480fc-8af7-455b-ba04-bcffa76f928d admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:13.767101 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e39742-8a2d-4012-84b3-c10d6e9dc8c3 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:13.767518 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e39742-8a2d-4012-84b3-c10d6e9dc8c3 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:13.767792 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e39742-8a2d-4012-84b3-c10d6e9dc8c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:13.786844 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-33e39742-8a2d-4012-84b3-c10d6e9dc8c3 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:13.788263 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-33e39742-8a2d-4012-84b3-c10d6e9dc8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:13.830393 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795480fc-8af7-455b-ba04-bcffa76f928d admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:13.836856 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae76bd-a06a-4ec3-ab7d-23de03d5e8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:13.837496 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae76bd-a06a-4ec3-ab7d-23de03d5e8b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:13.837771 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae76bd-a06a-4ec3-ab7d-23de03d5e8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:13.838050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae76bd-a06a-4ec3-ab7d-23de03d5e8b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:13.838719 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae76bd-a06a-4ec3-ab7d-23de03d5e8b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:13.840798 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae76bd-a06a-4ec3-ab7d-23de03d5e8b2 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:13.849446 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1259/2489] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:21:13.860295 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e39742-8a2d-4012-84b3-c10d6e9dc8c3 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:13.861726 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-33e39742-8a2d-4012-84b3-c10d6e9dc8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:13.864211 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1231/2490] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:13 2022] POST /identity/v3/users => generated 307 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:21:13.868126 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae76bd-a06a-4ec3-ab7d-23de03d5e8b2 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:13.878240 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae76bd-a06a-4ec3-ab7d-23de03d5e8b2 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:13.879507 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:13.884261 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf70a666-d8a8-439e-8c18-468927a8735e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:13.885202 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70a666-d8a8-439e-8c18-468927a8735e admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:13.885488 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70a666-d8a8-439e-8c18-468927a8735e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:13.885801 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70a666-d8a8-439e-8c18-468927a8735e admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:13.886688 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf70a666-d8a8-439e-8c18-468927a8735e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:13.888356 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf70a666-d8a8-439e-8c18-468927a8735e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:13.898933 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1232/2491] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:21:13.902989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf70a666-d8a8-439e-8c18-468927a8735e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:13.914587 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf70a666-d8a8-439e-8c18-468927a8735e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:13.923069 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94fc36a-0eef-44e7-9dea-8250b8114455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:13.923751 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fc36a-0eef-44e7-9dea-8250b8114455 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:13.924037 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fc36a-0eef-44e7-9dea-8250b8114455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:13.924321 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fc36a-0eef-44e7-9dea-8250b8114455 admin admin] PATH_INFO: `/v3/projects/bd937098290647aca1b5ed2a868d622f/users/dec065bcc1d841c8b3fe3cd62031e419/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:13.933657 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:13.936875 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e39742-8a2d-4012-84b3-c10d6e9dc8c3 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:13.941084 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f03ac-d630-4bfe-bc58-18e04e5d48a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:13.941950 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f03ac-d630-4bfe-bc58-18e04e5d48a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:13.942189 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94fc36a-0eef-44e7-9dea-8250b8114455 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd937098290647aca1b5ed2a868d622f, user_id=dec065bcc1d841c8b3fe3cd62031e419, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:13.942297 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f03ac-d630-4bfe-bc58-18e04e5d48a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:13.942521 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f03ac-d630-4bfe-bc58-18e04e5d48a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:13.945890 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94fc36a-0eef-44e7-9dea-8250b8114455 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:13.957674 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f03ac-d630-4bfe-bc58-18e04e5d48a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:13.959109 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f03ac-d630-4bfe-bc58-18e04e5d48a7 service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:13.960515 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1233/2492] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:13 2022] PUT /identity/v3/projects/bd937098290647aca1b5ed2a868d622f/users/dec065bcc1d841c8b3fe3cd62031e419/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:13.964416 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94fc36a-0eef-44e7-9dea-8250b8114455 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:13.971326 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1260/2493] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:21:14.028878 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94fc36a-0eef-44e7-9dea-8250b8114455 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:14.033795 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf1b2b-3dc7-415c-b496-b9ef1ba53f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:14.034417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf1b2b-3dc7-415c-b496-b9ef1ba53f49 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:14.034709 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf1b2b-3dc7-415c-b496-b9ef1ba53f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:14.034992 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf1b2b-3dc7-415c-b496-b9ef1ba53f49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:14.035697 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf1b2b-3dc7-415c-b496-b9ef1ba53f49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:14.037418 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf1b2b-3dc7-415c-b496-b9ef1ba53f49 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:14.048263 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1234/2494] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:21:14.052215 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f03ac-d630-4bfe-bc58-18e04e5d48a7 service neutron] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:14.063150 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f03ac-d630-4bfe-bc58-18e04e5d48a7 service neutron] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:14.068027 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc34fbe-b7af-476d-b782-844587bc3c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:14.068945 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc34fbe-b7af-476d-b782-844587bc3c22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:14.069838 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc34fbe-b7af-476d-b782-844587bc3c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:14.070106 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc34fbe-b7af-476d-b782-844587bc3c22 admin admin] PATH_INFO: `/v3/projects/bd937098290647aca1b5ed2a868d622f/users/dec065bcc1d841c8b3fe3cd62031e419/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:14.072100 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc34fbe-b7af-476d-b782-844587bc3c22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd937098290647aca1b5ed2a868d622f, user_id=dec065bcc1d841c8b3fe3cd62031e419, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:14.074778 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc34fbe-b7af-476d-b782-844587bc3c22 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:14.085645 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1261/2495] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:14 2022] PUT /identity/v3/projects/bd937098290647aca1b5ed2a868d622f/users/dec065bcc1d841c8b3fe3cd62031e419/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:14.096144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454982e-30c4-413c-824b-715e922dd233 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:14.096440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454982e-30c4-413c-824b-715e922dd233 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:14.096760 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454982e-30c4-413c-824b-715e922dd233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:14.126204 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-3454982e-30c4-413c-824b-715e922dd233 None None] MFA Rules not processed for user `dec065bcc1d841c8b3fe3cd62031e419`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:14.127426 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3454982e-30c4-413c-824b-715e922dd233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:14.200214 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1235/2496] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:14 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:14.215487 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc34fbe-b7af-476d-b782-844587bc3c22 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:14.217464 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-fbc34fbe-b7af-476d-b782-844587bc3c22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:14.282696 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc34fbe-b7af-476d-b782-844587bc3c22 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:14.286709 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bdd057-b158-4327-ba25-d0fffc4f88b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:14.287347 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bdd057-b158-4327-ba25-d0fffc4f88b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:14.287676 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bdd057-b158-4327-ba25-d0fffc4f88b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:14.287982 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bdd057-b158-4327-ba25-d0fffc4f88b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:14.298715 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bdd057-b158-4327-ba25-d0fffc4f88b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:14.300345 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bdd057-b158-4327-ba25-d0fffc4f88b0 service cinder] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:14.317648 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1262/2497] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:14 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:14.476691 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454982e-30c4-413c-824b-715e922dd233 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:14.544623 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454982e-30c4-413c-824b-715e922dd233 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:14.549014 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c7a22-f493-4ab6-b99a-ef0ae27b2c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:14.549658 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c7a22-f493-4ab6-b99a-ef0ae27b2c4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:14.550363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c7a22-f493-4ab6-b99a-ef0ae27b2c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:14.550363 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c7a22-f493-4ab6-b99a-ef0ae27b2c4c admin admin] PATH_INFO: `/v3/users/41561005400b4b07aee87c425c15d5e8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:14.551378 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c7a22-f493-4ab6-b99a-ef0ae27b2c4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=41561005400b4b07aee87c425c15d5e8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:14.553477 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c7a22-f493-4ab6-b99a-ef0ae27b2c4c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:14.565490 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:14.605328 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-686c7a22-f493-4ab6-b99a-ef0ae27b2c4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41561005400b4b07aee87c425c15d5e8'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:14.612777 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-686c7a22-f493-4ab6-b99a-ef0ae27b2c4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41561005400b4b07aee87c425c15d5e8'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:14.613554 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-686c7a22-f493-4ab6-b99a-ef0ae27b2c4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41561005400b4b07aee87c425c15d5e8'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:14.622944 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-686c7a22-f493-4ab6-b99a-ef0ae27b2c4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41561005400b4b07aee87c425c15d5e8'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:14.623698 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-686c7a22-f493-4ab6-b99a-ef0ae27b2c4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41561005400b4b07aee87c425c15d5e8'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:14.637148 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0cffdf-cf87-46fd-892e-b2bc84d4690e None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:14.637583 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0cffdf-cf87-46fd-892e-b2bc84d4690e None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:14.637775 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0cffdf-cf87-46fd-892e-b2bc84d4690e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:14.656701 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1236/2498] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:14 2022] DELETE /identity/v3/users/41561005400b4b07aee87c425c15d5e8 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:14.661642 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a37806-3bd3-4e03-85df-85d097fd990d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:14.661999 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a37806-3bd3-4e03-85df-85d097fd990d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:14.662345 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a37806-3bd3-4e03-85df-85d097fd990d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:14.691193 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-41a37806-3bd3-4e03-85df-85d097fd990d None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:14.692730 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-41a37806-3bd3-4e03-85df-85d097fd990d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:14.704681 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-5a0cffdf-cf87-46fd-892e-b2bc84d4690e None None] MFA Rules not processed for user `6c39041deecc4da0ac3648636b9eff86`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:14.705923 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-5a0cffdf-cf87-46fd-892e-b2bc84d4690e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:14.781416 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1237/2499] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:14.790899 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1263/2500] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:14.795485 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a37806-3bd3-4e03-85df-85d097fd990d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:14.801703 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-41a37806-3bd3-4e03-85df-85d097fd990d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:14.802881 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0cffdf-cf87-46fd-892e-b2bc84d4690e None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:14.806698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-5a0cffdf-cf87-46fd-892e-b2bc84d4690e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:14.842560 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0cffdf-cf87-46fd-892e-b2bc84d4690e None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:14.850327 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62900d4-14bb-4454-bb2d-50846a01dd0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:14.851449 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62900d4-14bb-4454-bb2d-50846a01dd0f admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:14.851861 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62900d4-14bb-4454-bb2d-50846a01dd0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:14.852216 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62900d4-14bb-4454-bb2d-50846a01dd0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:14.853276 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62900d4-14bb-4454-bb2d-50846a01dd0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:14.855870 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62900d4-14bb-4454-bb2d-50846a01dd0f admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:14.876088 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a37806-3bd3-4e03-85df-85d097fd990d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:14.880578 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec968e-b348-4bf0-a3f9-3c7d221416ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:14.882573 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec968e-b348-4bf0-a3f9-3c7d221416ac service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:14.882933 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec968e-b348-4bf0-a3f9-3c7d221416ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:14.883476 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec968e-b348-4bf0-a3f9-3c7d221416ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:14.896280 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ec968e-b348-4bf0-a3f9-3c7d221416ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:14.897836 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ec968e-b348-4bf0-a3f9-3c7d221416ac service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:14.902548 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1264/2501] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:14 2022] POST /identity/v3/projects => generated 405 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:14.909405 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1238/2502] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:21:14.916779 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62900d4-14bb-4454-bb2d-50846a01dd0f admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:14.933516 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:15.000978 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62900d4-14bb-4454-bb2d-50846a01dd0f admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:15.009523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b602f9b-f5be-4131-86d6-c45f5eda01e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:15.010329 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b602f9b-f5be-4131-86d6-c45f5eda01e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:15.010683 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b602f9b-f5be-4131-86d6-c45f5eda01e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:15.011148 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b602f9b-f5be-4131-86d6-c45f5eda01e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:15.011834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b602f9b-f5be-4131-86d6-c45f5eda01e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:15.013884 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b602f9b-f5be-4131-86d6-c45f5eda01e1 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:15.041215 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1265/2503] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:14 2022] POST /identity/v3/users => generated 309 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:15.042461 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec968e-b348-4bf0-a3f9-3c7d221416ac service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:15.060731 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec968e-b348-4bf0-a3f9-3c7d221416ac service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:15.068912 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c4d9b-05ac-429a-92d2-ebbbb5299c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:15.070237 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c4d9b-05ac-429a-92d2-ebbbb5299c7f admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:15.070522 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c4d9b-05ac-429a-92d2-ebbbb5299c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:15.070828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c4d9b-05ac-429a-92d2-ebbbb5299c7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:15.071560 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887c4d9b-05ac-429a-92d2-ebbbb5299c7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:15.073453 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887c4d9b-05ac-429a-92d2-ebbbb5299c7f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:15.101870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1239/2504] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:15.103832 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b602f9b-f5be-4131-86d6-c45f5eda01e1 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:15.115501 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b602f9b-f5be-4131-86d6-c45f5eda01e1 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:15.121998 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d381e9-2e08-4452-972c-655e1bb02325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:15.122993 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d381e9-2e08-4452-972c-655e1bb02325 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:15.123491 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d381e9-2e08-4452-972c-655e1bb02325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:15.123942 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d381e9-2e08-4452-972c-655e1bb02325 admin admin] PATH_INFO: `/v3/projects/f371ea7a7be1403b933eca23ff10e968/users/809c1fcc335944cda6ebe50f5c62368f/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:15.142122 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d381e9-2e08-4452-972c-655e1bb02325 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f371ea7a7be1403b933eca23ff10e968, user_id=809c1fcc335944cda6ebe50f5c62368f, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:15.144733 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d381e9-2e08-4452-972c-655e1bb02325 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:15.164540 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1266/2505] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:15 2022] PUT /identity/v3/projects/f371ea7a7be1403b933eca23ff10e968/users/809c1fcc335944cda6ebe50f5c62368f/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:15.168704 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c4d9b-05ac-429a-92d2-ebbbb5299c7f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:15.180177 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c4d9b-05ac-429a-92d2-ebbbb5299c7f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:15.201768 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:15.233614 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d381e9-2e08-4452-972c-655e1bb02325 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:15.236762 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-75d381e9-2e08-4452-972c-655e1bb02325 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:15.265390 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd0c90-c795-4bee-9acb-3507547c550e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:15.266026 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd0c90-c795-4bee-9acb-3507547c550e admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:15.266303 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd0c90-c795-4bee-9acb-3507547c550e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:15.266581 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd0c90-c795-4bee-9acb-3507547c550e admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:15.267289 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cd0c90-c795-4bee-9acb-3507547c550e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:15.269104 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cd0c90-c795-4bee-9acb-3507547c550e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:15.279448 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1240/2506] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:15.283641 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd0c90-c795-4bee-9acb-3507547c550e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:15.294192 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd0c90-c795-4bee-9acb-3507547c550e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:15.299140 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf7c98-214e-44c0-8854-50bdc12d75a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:15.299866 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf7c98-214e-44c0-8854-50bdc12d75a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:15.300023 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf7c98-214e-44c0-8854-50bdc12d75a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:15.300303 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf7c98-214e-44c0-8854-50bdc12d75a2 admin admin] PATH_INFO: `/v3/projects/f371ea7a7be1403b933eca23ff10e968/users/809c1fcc335944cda6ebe50f5c62368f/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:15.302343 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baf7c98-214e-44c0-8854-50bdc12d75a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f371ea7a7be1403b933eca23ff10e968, user_id=809c1fcc335944cda6ebe50f5c62368f, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:15.305749 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baf7c98-214e-44c0-8854-50bdc12d75a2 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:15.313903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d381e9-2e08-4452-972c-655e1bb02325 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:15.317648 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1241/2507] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:15 2022] PUT /identity/v3/projects/f371ea7a7be1403b933eca23ff10e968/users/809c1fcc335944cda6ebe50f5c62368f/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:15.373548 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fe1ea-d576-4e79-888b-488c175de991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:15.374282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fe1ea-d576-4e79-888b-488c175de991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:15.374572 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fe1ea-d576-4e79-888b-488c175de991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:15.374853 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fe1ea-d576-4e79-888b-488c175de991 admin admin] PATH_INFO: `/v3/projects/bbad1c32ba0b497bb7e0c39645d76ff3` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:15.376051 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045fe1ea-d576-4e79-888b-488c175de991 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbad1c32ba0b497bb7e0c39645d76ff3)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:15.378044 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045fe1ea-d576-4e79-888b-488c175de991 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:15.412050 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:15.432455 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-045fe1ea-d576-4e79-888b-488c175de991 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbad1c32ba0b497bb7e0c39645d76ff3', 'request_id': 'req-045fe1ea-d576-4e79-888b-488c175de991'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:15.439561 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-045fe1ea-d576-4e79-888b-488c175de991 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbad1c32ba0b497bb7e0c39645d76ff3', 'request_id': 'req-045fe1ea-d576-4e79-888b-488c175de991'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:15.446303 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-045fe1ea-d576-4e79-888b-488c175de991 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbad1c32ba0b497bb7e0c39645d76ff3', 'request_id': 'req-045fe1ea-d576-4e79-888b-488c175de991'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:15.458968 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-045fe1ea-d576-4e79-888b-488c175de991 admin admin] The token cache is being invalidate because project bbad1c32ba0b497bb7e0c39645d76ff3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:15.459261 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-045fe1ea-d576-4e79-888b-488c175de991 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:15.459951 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-045fe1ea-d576-4e79-888b-488c175de991 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:15.461576 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1267/2508] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:15 2022] DELETE /identity/v3/projects/bbad1c32ba0b497bb7e0c39645d76ff3 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:15.465715 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf7c98-214e-44c0-8854-50bdc12d75a2 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:15.468026 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6baf7c98-214e-44c0-8854-50bdc12d75a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:15.482358 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf7c98-214e-44c0-8854-50bdc12d75a2 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:15.541993 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4695a337-7ff9-4cda-9910-9f4dbdf5681b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:15.542657 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695a337-7ff9-4cda-9910-9f4dbdf5681b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:15.542951 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695a337-7ff9-4cda-9910-9f4dbdf5681b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:15.543233 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695a337-7ff9-4cda-9910-9f4dbdf5681b admin admin] PATH_INFO: `/v3/users/dec065bcc1d841c8b3fe3cd62031e419` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:15.544424 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4695a337-7ff9-4cda-9910-9f4dbdf5681b admin admin] RBAC: Authorizing `identity:delete_user(user_id=dec065bcc1d841c8b3fe3cd62031e419)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:15.546394 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4695a337-7ff9-4cda-9910-9f4dbdf5681b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:15.595608 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4695a337-7ff9-4cda-9910-9f4dbdf5681b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dec065bcc1d841c8b3fe3cd62031e419'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:15.602629 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4695a337-7ff9-4cda-9910-9f4dbdf5681b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dec065bcc1d841c8b3fe3cd62031e419'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:15.603312 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4695a337-7ff9-4cda-9910-9f4dbdf5681b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dec065bcc1d841c8b3fe3cd62031e419'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:15.610874 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4695a337-7ff9-4cda-9910-9f4dbdf5681b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dec065bcc1d841c8b3fe3cd62031e419'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:15.611957 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4695a337-7ff9-4cda-9910-9f4dbdf5681b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dec065bcc1d841c8b3fe3cd62031e419'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:15.635445 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1242/2509] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:15 2022] DELETE /identity/v3/users/dec065bcc1d841c8b3fe3cd62031e419 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:15.936351 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fe1ea-d576-4e79-888b-488c175de991 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:15.948820 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fe1ea-d576-4e79-888b-488c175de991 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:15.966724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7790817-371f-4229-afb7-89d67d767b6f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:15.968255 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7790817-371f-4229-afb7-89d67d767b6f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:15.968880 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7790817-371f-4229-afb7-89d67d767b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:16.014623 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f7790817-371f-4229-afb7-89d67d767b6f None None] MFA Rules not processed for user `809c1fcc335944cda6ebe50f5c62368f`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:16.015857 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f7790817-371f-4229-afb7-89d67d767b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:16.022743 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2461fcf5-c517-4f17-ab50-ec5287ee811d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:16.023467 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461fcf5-c517-4f17-ab50-ec5287ee811d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:16.023751 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461fcf5-c517-4f17-ab50-ec5287ee811d admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:16.024028 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461fcf5-c517-4f17-ab50-ec5287ee811d admin admin] PATH_INFO: `/v3/projects/bd937098290647aca1b5ed2a868d622f` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:16.025214 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2461fcf5-c517-4f17-ab50-ec5287ee811d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd937098290647aca1b5ed2a868d622f)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:16.026675 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:16.027114 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2461fcf5-c517-4f17-ab50-ec5287ee811d admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:16.083287 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-2461fcf5-c517-4f17-ab50-ec5287ee811d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd937098290647aca1b5ed2a868d622f', 'request_id': 'req-2461fcf5-c517-4f17-ab50-ec5287ee811d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:16.091222 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-2461fcf5-c517-4f17-ab50-ec5287ee811d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd937098290647aca1b5ed2a868d622f', 'request_id': 'req-2461fcf5-c517-4f17-ab50-ec5287ee811d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:16.094170 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1243/2510] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:15 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:21:16.099659 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-2461fcf5-c517-4f17-ab50-ec5287ee811d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd937098290647aca1b5ed2a868d622f', 'request_id': 'req-2461fcf5-c517-4f17-ab50-ec5287ee811d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:16.114776 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-2461fcf5-c517-4f17-ab50-ec5287ee811d admin admin] The token cache is being invalidate because project bd937098290647aca1b5ed2a868d622f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:16.115132 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-2461fcf5-c517-4f17-ab50-ec5287ee811d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:16.115848 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-2461fcf5-c517-4f17-ab50-ec5287ee811d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:16.117927 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1268/2511] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:15 2022] DELETE /identity/v3/projects/bd937098290647aca1b5ed2a868d622f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:16.122951 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05896be9-af08-4d43-bf1e-46d0725bb368 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:16.123243 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05896be9-af08-4d43-bf1e-46d0725bb368 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:16.123550 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05896be9-af08-4d43-bf1e-46d0725bb368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:16.147293 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-05896be9-af08-4d43-bf1e-46d0725bb368 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:16.148582 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-05896be9-af08-4d43-bf1e-46d0725bb368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:16.225858 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1244/2512] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:16.233615 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24e576-1a48-4faf-80cc-99566c48c925 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:16.233910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24e576-1a48-4faf-80cc-99566c48c925 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:16.234198 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24e576-1a48-4faf-80cc-99566c48c925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:16.258593 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-0f24e576-1a48-4faf-80cc-99566c48c925 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:16.260187 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-0f24e576-1a48-4faf-80cc-99566c48c925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:16.268690 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1269/2513] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:16.273376 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05896be9-af08-4d43-bf1e-46d0725bb368 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:16.283690 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05896be9-af08-4d43-bf1e-46d0725bb368 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:16.289330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f6db8f-d082-417f-9a53-4cc26e29cf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:16.290000 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6db8f-d082-417f-9a53-4cc26e29cf74 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:16.290290 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6db8f-d082-417f-9a53-4cc26e29cf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:16.290851 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6db8f-d082-417f-9a53-4cc26e29cf74 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:16.291302 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f6db8f-d082-417f-9a53-4cc26e29cf74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:21:16.291609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f6db8f-d082-417f-9a53-4cc26e29cf74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:16.293546 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f6db8f-d082-417f-9a53-4cc26e29cf74 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:16.307302 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1245/2514] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:21:16.313098 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f24e576-1a48-4faf-80cc-99566c48c925 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:16.325980 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f24e576-1a48-4faf-80cc-99566c48c925 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:16.331995 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99275745-4c06-422f-b968-acb9e3fafd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:16.332711 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99275745-4c06-422f-b968-acb9e3fafd4a admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:16.333002 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99275745-4c06-422f-b968-acb9e3fafd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:16.333445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99275745-4c06-422f-b968-acb9e3fafd4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:16.334056 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99275745-4c06-422f-b968-acb9e3fafd4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:16.336462 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99275745-4c06-422f-b968-acb9e3fafd4a admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:16.378456 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1270/2515] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:16 2022] POST /identity/v3/projects => generated 365 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:21:16.382125 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f6db8f-d082-417f-9a53-4cc26e29cf74 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:16.392105 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f6db8f-d082-417f-9a53-4cc26e29cf74 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:16.472946 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb59b414-cf5c-411a-8f94-7da9858213bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:16.473573 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59b414-cf5c-411a-8f94-7da9858213bf admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:16.473856 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59b414-cf5c-411a-8f94-7da9858213bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:16.474129 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59b414-cf5c-411a-8f94-7da9858213bf admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:16.476013 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb59b414-cf5c-411a-8f94-7da9858213bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:16.478010 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb59b414-cf5c-411a-8f94-7da9858213bf admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:16.509976 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1246/2516] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:16 2022] POST /identity/v3/users => generated 283 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:16.515515 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99275745-4c06-422f-b968-acb9e3fafd4a admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:16.528618 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99275745-4c06-422f-b968-acb9e3fafd4a admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:16.534578 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb6b7b-4888-446f-b230-696ec57bf3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:16.535304 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb6b7b-4888-446f-b230-696ec57bf3f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:16.535718 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb6b7b-4888-446f-b230-696ec57bf3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:16.536006 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb6b7b-4888-446f-b230-696ec57bf3f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:16.536780 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb6b7b-4888-446f-b230-696ec57bf3f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:16.538418 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb6b7b-4888-446f-b230-696ec57bf3f3 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:16.549738 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1271/2517] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:21:16.554476 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb59b414-cf5c-411a-8f94-7da9858213bf admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:16.565926 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb59b414-cf5c-411a-8f94-7da9858213bf admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:16.571598 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa706b84-f740-44c7-9242-e5a2b0f4bddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:16.572359 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa706b84-f740-44c7-9242-e5a2b0f4bddc admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:16.572918 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa706b84-f740-44c7-9242-e5a2b0f4bddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:16.573288 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa706b84-f740-44c7-9242-e5a2b0f4bddc admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:16.574790 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa706b84-f740-44c7-9242-e5a2b0f4bddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:16.576594 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa706b84-f740-44c7-9242-e5a2b0f4bddc admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:16.592768 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb6b7b-4888-446f-b230-696ec57bf3f3 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:16.593198 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1247/2518] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:21:16.613086 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb6b7b-4888-446f-b230-696ec57bf3f3 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:16.619915 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0bf6a-20e6-40a9-b19e-7a041dc3729e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:16.620684 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0bf6a-20e6-40a9-b19e-7a041dc3729e admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:16.620972 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0bf6a-20e6-40a9-b19e-7a041dc3729e admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:16.621257 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0bf6a-20e6-40a9-b19e-7a041dc3729e admin admin] PATH_INFO: `/v3/projects/03dd0ca268454b3eaf735899752e6d33/users/26a31e77b19c4ba6bf3a003407304b6a/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:16.629277 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:16.664687 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da0bf6a-20e6-40a9-b19e-7a041dc3729e admin admin] RBAC: Authorizing `identity:create_grant(project_id=03dd0ca268454b3eaf735899752e6d33, user_id=26a31e77b19c4ba6bf3a003407304b6a, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:16.667878 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da0bf6a-20e6-40a9-b19e-7a041dc3729e admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:16.694815 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1272/2519] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:16 2022] PUT /identity/v3/projects/03dd0ca268454b3eaf735899752e6d33/users/26a31e77b19c4ba6bf3a003407304b6a/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:16.706411 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba57c5-1dad-4153-92f1-53a50c01f311 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:16.706856 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba57c5-1dad-4153-92f1-53a50c01f311 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:16.707142 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba57c5-1dad-4153-92f1-53a50c01f311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:16.737682 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:16.741753 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-29ba57c5-1dad-4153-92f1-53a50c01f311 None None] MFA Rules not processed for user `26a31e77b19c4ba6bf3a003407304b6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:16.744991 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-29ba57c5-1dad-4153-92f1-53a50c01f311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:16.824088 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1248/2520] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:16 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:16.828831 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0bf6a-20e6-40a9-b19e-7a041dc3729e admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:16.893527 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0bf6a-20e6-40a9-b19e-7a041dc3729e admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:16.899028 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b2029d-b51f-47d5-a50e-7abd031de3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:16.899705 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2029d-b51f-47d5-a50e-7abd031de3dc admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:16.900106 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2029d-b51f-47d5-a50e-7abd031de3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:16.900393 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2029d-b51f-47d5-a50e-7abd031de3dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:16.901138 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b2029d-b51f-47d5-a50e-7abd031de3dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:16.903029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b2029d-b51f-47d5-a50e-7abd031de3dc admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:16.929858 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1273/2521] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:16 2022] POST /identity/v3/projects => generated 363 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:16.933718 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ba57c5-1dad-4153-92f1-53a50c01f311 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:16.945471 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ba57c5-1dad-4153-92f1-53a50c01f311 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:17.007611 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48940fb1-1b74-4e46-b481-abf6094e4da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:17.008262 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48940fb1-1b74-4e46-b481-abf6094e4da3 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:17.008565 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48940fb1-1b74-4e46-b481-abf6094e4da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:17.009346 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48940fb1-1b74-4e46-b481-abf6094e4da3 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:17.010045 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48940fb1-1b74-4e46-b481-abf6094e4da3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:17.012005 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48940fb1-1b74-4e46-b481-abf6094e4da3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:17.034212 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1249/2522] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:16 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:17.039264 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b2029d-b51f-47d5-a50e-7abd031de3dc admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:17.049633 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b2029d-b51f-47d5-a50e-7abd031de3dc admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:17.054881 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d615e3-5a82-45cc-8468-d6cc0a1a6c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:17.055508 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d615e3-5a82-45cc-8468-d6cc0a1a6c76 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:17.055803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d615e3-5a82-45cc-8468-d6cc0a1a6c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:17.056077 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d615e3-5a82-45cc-8468-d6cc0a1a6c76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:17.056801 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d615e3-5a82-45cc-8468-d6cc0a1a6c76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:17.058397 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d615e3-5a82-45cc-8468-d6cc0a1a6c76 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:17.070378 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1274/2523] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:21:17.074732 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48940fb1-1b74-4e46-b481-abf6094e4da3 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:17.085277 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48940fb1-1b74-4e46-b481-abf6094e4da3 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:17.089931 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff81f7-0e7f-4bc9-a3bd-447b628a2d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:17.090603 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff81f7-0e7f-4bc9-a3bd-447b628a2d41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:17.090919 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff81f7-0e7f-4bc9-a3bd-447b628a2d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:17.091199 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff81f7-0e7f-4bc9-a3bd-447b628a2d41 admin admin] PATH_INFO: `/v3/projects/47dd527f45dc4c1ba781e99773545627/users/b9155469bda84d2f93f619d1e7c9fa8c/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:17.106420 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ff81f7-0e7f-4bc9-a3bd-447b628a2d41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47dd527f45dc4c1ba781e99773545627, user_id=b9155469bda84d2f93f619d1e7c9fa8c, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:17.109582 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ff81f7-0e7f-4bc9-a3bd-447b628a2d41 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:17.123193 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1250/2524] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:17 2022] PUT /identity/v3/projects/47dd527f45dc4c1ba781e99773545627/users/b9155469bda84d2f93f619d1e7c9fa8c/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:17.127316 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d615e3-5a82-45cc-8468-d6cc0a1a6c76 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:17.138470 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d615e3-5a82-45cc-8468-d6cc0a1a6c76 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:17.198127 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733328da-99d1-44c4-92f3-ae09587395e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:17.199139 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733328da-99d1-44c4-92f3-ae09587395e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:17.199139 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733328da-99d1-44c4-92f3-ae09587395e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:17.199363 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733328da-99d1-44c4-92f3-ae09587395e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:17.200023 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733328da-99d1-44c4-92f3-ae09587395e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:17.201906 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733328da-99d1-44c4-92f3-ae09587395e6 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:17.213171 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1275/2525] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:17.216102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff81f7-0e7f-4bc9-a3bd-447b628a2d41 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:17.227038 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff81f7-0e7f-4bc9-a3bd-447b628a2d41 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:17.231667 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4537e46-6923-4fb5-9a01-e65722c5e6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:17.232329 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4537e46-6923-4fb5-9a01-e65722c5e6bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:17.232643 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4537e46-6923-4fb5-9a01-e65722c5e6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:17.232922 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4537e46-6923-4fb5-9a01-e65722c5e6bd admin admin] PATH_INFO: `/v3/projects/47dd527f45dc4c1ba781e99773545627/users/b9155469bda84d2f93f619d1e7c9fa8c/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:17.234911 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4537e46-6923-4fb5-9a01-e65722c5e6bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=47dd527f45dc4c1ba781e99773545627, user_id=b9155469bda84d2f93f619d1e7c9fa8c, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:17.237575 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4537e46-6923-4fb5-9a01-e65722c5e6bd admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:17.248723 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1251/2526] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:17 2022] PUT /identity/v3/projects/47dd527f45dc4c1ba781e99773545627/users/b9155469bda84d2f93f619d1e7c9fa8c/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:17.259658 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72ea06-35ff-4eca-a502-bcb307d2589e None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:17.259955 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72ea06-35ff-4eca-a502-bcb307d2589e None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:17.260240 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72ea06-35ff-4eca-a502-bcb307d2589e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:17.292408 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4537e46-6923-4fb5-9a01-e65722c5e6bd admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:17.293273 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-ea72ea06-35ff-4eca-a502-bcb307d2589e None None] MFA Rules not processed for user `b9155469bda84d2f93f619d1e7c9fa8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:17.294197 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e4537e46-6923-4fb5-9a01-e65722c5e6bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:17.294471 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-ea72ea06-35ff-4eca-a502-bcb307d2589e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:17.340615 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:17.359901 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:17.369583 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4537e46-6923-4fb5-9a01-e65722c5e6bd admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:17.370926 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1276/2527] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:17 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:17.374558 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88015e9a-9063-4032-bf8a-8b34d63cf9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:17.375217 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88015e9a-9063-4032-bf8a-8b34d63cf9f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:17.375499 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88015e9a-9063-4032-bf8a-8b34d63cf9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:17.375776 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88015e9a-9063-4032-bf8a-8b34d63cf9f5 admin admin] PATH_INFO: `/v3/users/6c39041deecc4da0ac3648636b9eff86` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:17.377238 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88015e9a-9063-4032-bf8a-8b34d63cf9f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c39041deecc4da0ac3648636b9eff86)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:17.379187 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88015e9a-9063-4032-bf8a-8b34d63cf9f5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:17.382807 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea72ea06-35ff-4eca-a502-bcb307d2589e None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:17.384282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-ea72ea06-35ff-4eca-a502-bcb307d2589e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:17.427148 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-88015e9a-9063-4032-bf8a-8b34d63cf9f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c39041deecc4da0ac3648636b9eff86'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:17.435298 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-88015e9a-9063-4032-bf8a-8b34d63cf9f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c39041deecc4da0ac3648636b9eff86'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:17.436060 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-88015e9a-9063-4032-bf8a-8b34d63cf9f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c39041deecc4da0ac3648636b9eff86'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:17.443195 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea72ea06-35ff-4eca-a502-bcb307d2589e None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:17.446156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-88015e9a-9063-4032-bf8a-8b34d63cf9f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c39041deecc4da0ac3648636b9eff86'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:17.446874 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-88015e9a-9063-4032-bf8a-8b34d63cf9f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c39041deecc4da0ac3648636b9eff86'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:17.447381 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e21eb-b766-4d9f-a5f2-2ac6f5906bff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:17.448000 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e21eb-b766-4d9f-a5f2-2ac6f5906bff service cinder] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:17.448283 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e21eb-b766-4d9f-a5f2-2ac6f5906bff service cinder] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:17.448590 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e21eb-b766-4d9f-a5f2-2ac6f5906bff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:17.450551 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-0c7e21eb-b766-4d9f-a5f2-2ac6f5906bff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:17.462488 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7e21eb-b766-4d9f-a5f2-2ac6f5906bff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:17.463996 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7e21eb-b766-4d9f-a5f2-2ac6f5906bff service cinder] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:17.464468 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1252/2528] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:17 2022] DELETE /identity/v3/users/6c39041deecc4da0ac3648636b9eff86 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:17.477633 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1277/2529] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:17 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:21:17.558470 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88015e9a-9063-4032-bf8a-8b34d63cf9f5 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:17.560323 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-88015e9a-9063-4032-bf8a-8b34d63cf9f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:17.618409 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88015e9a-9063-4032-bf8a-8b34d63cf9f5 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:17.622959 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee910d-7e57-454d-91b3-d4631e5d40d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:17.623622 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee910d-7e57-454d-91b3-d4631e5d40d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:17.623917 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee910d-7e57-454d-91b3-d4631e5d40d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:17.624186 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee910d-7e57-454d-91b3-d4631e5d40d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:17.626181 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-53ee910d-7e57-454d-91b3-d4631e5d40d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:17.646231 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e21eb-b766-4d9f-a5f2-2ac6f5906bff service cinder] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:17.648030 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-0c7e21eb-b766-4d9f-a5f2-2ac6f5906bff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:17.665577 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e21eb-b766-4d9f-a5f2-2ac6f5906bff service cinder] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:17.700053 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ee910d-7e57-454d-91b3-d4631e5d40d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:17.701543 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ee910d-7e57-454d-91b3-d4631e5d40d6 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:17.711526 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1253/2530] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:17 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:17.733659 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f1559-1c40-41b3-8a87-3a1bdc735a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:17.734389 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f1559-1c40-41b3-8a87-3a1bdc735a27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:17.735345 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f1559-1c40-41b3-8a87-3a1bdc735a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:17.735658 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f1559-1c40-41b3-8a87-3a1bdc735a27 admin admin] PATH_INFO: `/v3/projects/017be9a7280a4c189ee057f05a0a79d7` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:17.737297 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36f1559-1c40-41b3-8a87-3a1bdc735a27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=017be9a7280a4c189ee057f05a0a79d7)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:17.739368 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36f1559-1c40-41b3-8a87-3a1bdc735a27 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:17.802048 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-b36f1559-1c40-41b3-8a87-3a1bdc735a27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '017be9a7280a4c189ee057f05a0a79d7', 'request_id': 'req-b36f1559-1c40-41b3-8a87-3a1bdc735a27'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:17.812228 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-b36f1559-1c40-41b3-8a87-3a1bdc735a27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '017be9a7280a4c189ee057f05a0a79d7', 'request_id': 'req-b36f1559-1c40-41b3-8a87-3a1bdc735a27'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:17.822079 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-b36f1559-1c40-41b3-8a87-3a1bdc735a27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '017be9a7280a4c189ee057f05a0a79d7', 'request_id': 'req-b36f1559-1c40-41b3-8a87-3a1bdc735a27'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:17.839834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-b36f1559-1c40-41b3-8a87-3a1bdc735a27 admin admin] The token cache is being invalidate because project 017be9a7280a4c189ee057f05a0a79d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:17.840312 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-b36f1559-1c40-41b3-8a87-3a1bdc735a27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:17.841185 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-b36f1559-1c40-41b3-8a87-3a1bdc735a27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:17.844138 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1278/2531] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:17 2022] DELETE /identity/v3/projects/017be9a7280a4c189ee057f05a0a79d7 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:17.852110 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee910d-7e57-454d-91b3-d4631e5d40d6 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:17.871084 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee910d-7e57-454d-91b3-d4631e5d40d6 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:17.922535 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:17.966081 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35c28fa-697e-480a-9a58-058153172b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:17.966838 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c28fa-697e-480a-9a58-058153172b26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:17.967132 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c28fa-697e-480a-9a58-058153172b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:17.967517 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c28fa-697e-480a-9a58-058153172b26 admin admin] PATH_INFO: `/v3/users/809c1fcc335944cda6ebe50f5c62368f` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:17.968701 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35c28fa-697e-480a-9a58-058153172b26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=809c1fcc335944cda6ebe50f5c62368f)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:17.970746 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35c28fa-697e-480a-9a58-058153172b26 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:18.027241 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a35c28fa-697e-480a-9a58-058153172b26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '809c1fcc335944cda6ebe50f5c62368f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:18.034657 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a35c28fa-697e-480a-9a58-058153172b26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '809c1fcc335944cda6ebe50f5c62368f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:18.035384 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a35c28fa-697e-480a-9a58-058153172b26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '809c1fcc335944cda6ebe50f5c62368f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:18.044226 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a35c28fa-697e-480a-9a58-058153172b26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '809c1fcc335944cda6ebe50f5c62368f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:18.044939 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a35c28fa-697e-480a-9a58-058153172b26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '809c1fcc335944cda6ebe50f5c62368f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:18.064308 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1254/2532] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:17 2022] DELETE /identity/v3/users/809c1fcc335944cda6ebe50f5c62368f => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:18.255615 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f1559-1c40-41b3-8a87-3a1bdc735a27 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:18.257903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-b36f1559-1c40-41b3-8a87-3a1bdc735a27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:18.272757 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f1559-1c40-41b3-8a87-3a1bdc735a27 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:18.281732 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:18.332669 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd278bfb-a038-490a-8494-bf3fbe6dd62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:18.333318 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd278bfb-a038-490a-8494-bf3fbe6dd62c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:18.333602 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd278bfb-a038-490a-8494-bf3fbe6dd62c admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:18.333883 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd278bfb-a038-490a-8494-bf3fbe6dd62c admin admin] PATH_INFO: `/v3/projects/f371ea7a7be1403b933eca23ff10e968` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:18.335594 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd278bfb-a038-490a-8494-bf3fbe6dd62c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f371ea7a7be1403b933eca23ff10e968)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:18.337527 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd278bfb-a038-490a-8494-bf3fbe6dd62c admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:18.384155 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-bd278bfb-a038-490a-8494-bf3fbe6dd62c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f371ea7a7be1403b933eca23ff10e968', 'request_id': 'req-bd278bfb-a038-490a-8494-bf3fbe6dd62c'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:18.390624 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-bd278bfb-a038-490a-8494-bf3fbe6dd62c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f371ea7a7be1403b933eca23ff10e968', 'request_id': 'req-bd278bfb-a038-490a-8494-bf3fbe6dd62c'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:18.395871 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-bd278bfb-a038-490a-8494-bf3fbe6dd62c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f371ea7a7be1403b933eca23ff10e968', 'request_id': 'req-bd278bfb-a038-490a-8494-bf3fbe6dd62c'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:18.409205 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-bd278bfb-a038-490a-8494-bf3fbe6dd62c admin admin] The token cache is being invalidate because project f371ea7a7be1403b933eca23ff10e968 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:18.409493 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-bd278bfb-a038-490a-8494-bf3fbe6dd62c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:18.410111 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-bd278bfb-a038-490a-8494-bf3fbe6dd62c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:18.411608 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1279/2533] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:18 2022] DELETE /identity/v3/projects/f371ea7a7be1403b933eca23ff10e968 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:18.416769 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a811a-dd74-4873-a9ab-d2905c55825f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:18.417067 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a811a-dd74-4873-a9ab-d2905c55825f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:18.417352 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a811a-dd74-4873-a9ab-d2905c55825f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:18.436584 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-da6a811a-dd74-4873-a9ab-d2905c55825f None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:18.437802 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-da6a811a-dd74-4873-a9ab-d2905c55825f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:18.497867 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1255/2534] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:21:18.505843 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748318a-2ca4-472e-a799-e15347d9e11d None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:18.506177 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748318a-2ca4-472e-a799-e15347d9e11d None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:18.506465 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748318a-2ca4-472e-a799-e15347d9e11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:18.531187 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-5748318a-2ca4-472e-a799-e15347d9e11d None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:18.532465 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-5748318a-2ca4-472e-a799-e15347d9e11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:18.541307 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1280/2535] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:18.545547 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a811a-dd74-4873-a9ab-d2905c55825f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:18.555392 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a811a-dd74-4873-a9ab-d2905c55825f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:18.560223 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1ca19a-cc1e-4634-8072-8a3e70a47e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:18.560979 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ca19a-cc1e-4634-8072-8a3e70a47e5c admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:18.561261 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ca19a-cc1e-4634-8072-8a3e70a47e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:18.561535 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ca19a-cc1e-4634-8072-8a3e70a47e5c admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:18.562226 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1ca19a-cc1e-4634-8072-8a3e70a47e5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:21:18.562530 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1ca19a-cc1e-4634-8072-8a3e70a47e5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:18.564100 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1ca19a-cc1e-4634-8072-8a3e70a47e5c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:18.575373 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1256/2536] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:21:18.579600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5748318a-2ca4-472e-a799-e15347d9e11d None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:18.589567 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5748318a-2ca4-472e-a799-e15347d9e11d None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:18.594260 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31a54c-5596-42d6-b5f7-2e2bd945a718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:18.594883 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31a54c-5596-42d6-b5f7-2e2bd945a718 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:18.595150 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31a54c-5596-42d6-b5f7-2e2bd945a718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:18.595428 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31a54c-5596-42d6-b5f7-2e2bd945a718 admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:18.596087 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f31a54c-5596-42d6-b5f7-2e2bd945a718 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:18.598065 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f31a54c-5596-42d6-b5f7-2e2bd945a718 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:18.628581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1281/2537] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:18 2022] POST /identity/v3/projects => generated 409 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:18.629688 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1ca19a-cc1e-4634-8072-8a3e70a47e5c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:18.639589 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1ca19a-cc1e-4634-8072-8a3e70a47e5c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:18.671937 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:18.702386 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54718c6a-be8c-4ce5-800b-daf86980a876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:18.703081 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54718c6a-be8c-4ce5-800b-daf86980a876 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:18.703404 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54718c6a-be8c-4ce5-800b-daf86980a876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:18.703719 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54718c6a-be8c-4ce5-800b-daf86980a876 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:18.704580 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54718c6a-be8c-4ce5-800b-daf86980a876 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:18.706614 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54718c6a-be8c-4ce5-800b-daf86980a876 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:18.728518 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1257/2538] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:18 2022] POST /identity/v3/users => generated 305 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:21:18.732719 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31a54c-5596-42d6-b5f7-2e2bd945a718 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:18.742961 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31a54c-5596-42d6-b5f7-2e2bd945a718 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:18.747929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb51b7-55ba-4fb3-994f-bb60880f2efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:18.748577 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb51b7-55ba-4fb3-994f-bb60880f2efd admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:18.748864 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb51b7-55ba-4fb3-994f-bb60880f2efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:18.749139 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb51b7-55ba-4fb3-994f-bb60880f2efd admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:18.749836 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb51b7-55ba-4fb3-994f-bb60880f2efd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:18.751421 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb51b7-55ba-4fb3-994f-bb60880f2efd admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:18.762162 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1282/2539] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:18.766023 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54718c6a-be8c-4ce5-800b-daf86980a876 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:18.777187 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54718c6a-be8c-4ce5-800b-daf86980a876 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:18.782004 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dce50a-d419-4d4d-97db-f22d8e0c41a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:18.782627 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dce50a-d419-4d4d-97db-f22d8e0c41a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:18.782915 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dce50a-d419-4d4d-97db-f22d8e0c41a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:18.783195 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dce50a-d419-4d4d-97db-f22d8e0c41a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:18.783894 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dce50a-d419-4d4d-97db-f22d8e0c41a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:18.785510 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dce50a-d419-4d4d-97db-f22d8e0c41a0 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:18.797024 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1258/2540] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:18.801444 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb51b7-55ba-4fb3-994f-bb60880f2efd admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:18.811915 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb51b7-55ba-4fb3-994f-bb60880f2efd admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:18.817019 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ffe528-e3a4-41fe-b19d-bd7f1e477f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:18.817709 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffe528-e3a4-41fe-b19d-bd7f1e477f7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:18.817997 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffe528-e3a4-41fe-b19d-bd7f1e477f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:18.818282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffe528-e3a4-41fe-b19d-bd7f1e477f7a admin admin] PATH_INFO: `/v3/projects/4de236911e6c4b7e820700b474b0814c/users/c96f29dc500445ffbd5a2f2af965e16e/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:18.832981 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ffe528-e3a4-41fe-b19d-bd7f1e477f7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4de236911e6c4b7e820700b474b0814c, user_id=c96f29dc500445ffbd5a2f2af965e16e, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:18.836137 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ffe528-e3a4-41fe-b19d-bd7f1e477f7a admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:18.849798 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1283/2541] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:18 2022] PUT /identity/v3/projects/4de236911e6c4b7e820700b474b0814c/users/c96f29dc500445ffbd5a2f2af965e16e/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:18.861603 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cfca34-2412-4547-807a-7ba6e7cc4263 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:18.861926 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cfca34-2412-4547-807a-7ba6e7cc4263 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:18.862210 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cfca34-2412-4547-807a-7ba6e7cc4263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:18.889738 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b6cfca34-2412-4547-807a-7ba6e7cc4263 None None] MFA Rules not processed for user `c96f29dc500445ffbd5a2f2af965e16e`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:18.890913 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b6cfca34-2412-4547-807a-7ba6e7cc4263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:18.963526 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1259/2542] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:18 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:18.968019 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ffe528-e3a4-41fe-b19d-bd7f1e477f7a admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:19.014669 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:19.039863 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ffe528-e3a4-41fe-b19d-bd7f1e477f7a admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:19.044649 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6d0a9-6cdb-470c-9efd-d493d6428f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:19.045275 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6d0a9-6cdb-470c-9efd-d493d6428f25 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:19.045564 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6d0a9-6cdb-470c-9efd-d493d6428f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:19.045846 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6d0a9-6cdb-470c-9efd-d493d6428f25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:19.046558 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b6d0a9-6cdb-470c-9efd-d493d6428f25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:19.049057 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b6d0a9-6cdb-470c-9efd-d493d6428f25 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:19.079295 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1284/2543] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:18 2022] POST /identity/v3/projects => generated 409 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:19.083481 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cfca34-2412-4547-807a-7ba6e7cc4263 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:19.095109 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cfca34-2412-4547-807a-7ba6e7cc4263 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:19.162302 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f664f9-1a96-4c99-bbff-4fa3326dd07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:19.163025 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f664f9-1a96-4c99-bbff-4fa3326dd07c admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:19.163306 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f664f9-1a96-4c99-bbff-4fa3326dd07c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:19.163579 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f664f9-1a96-4c99-bbff-4fa3326dd07c admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:19.164455 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f664f9-1a96-4c99-bbff-4fa3326dd07c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:19.166391 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f664f9-1a96-4c99-bbff-4fa3326dd07c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:19.191829 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1260/2544] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:19 2022] POST /identity/v3/users => generated 311 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:19.193141 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6d0a9-6cdb-470c-9efd-d493d6428f25 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:19.204585 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6d0a9-6cdb-470c-9efd-d493d6428f25 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:19.210586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2cf21-187c-4553-9d2b-77f8d9d79d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:19.211216 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2cf21-187c-4553-9d2b-77f8d9d79d61 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:19.211506 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2cf21-187c-4553-9d2b-77f8d9d79d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:19.211808 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2cf21-187c-4553-9d2b-77f8d9d79d61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:19.212530 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a2cf21-187c-4553-9d2b-77f8d9d79d61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:19.214122 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a2cf21-187c-4553-9d2b-77f8d9d79d61 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:19.226571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1285/2545] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:19.230872 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f664f9-1a96-4c99-bbff-4fa3326dd07c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:19.243386 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f664f9-1a96-4c99-bbff-4fa3326dd07c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:19.248648 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cc0414-cb95-4540-90f9-c389c5f4b12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:19.249487 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc0414-cb95-4540-90f9-c389c5f4b12a admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:19.249772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc0414-cb95-4540-90f9-c389c5f4b12a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:19.250056 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc0414-cb95-4540-90f9-c389c5f4b12a admin admin] PATH_INFO: `/v3/projects/828263d2c2134eba9781c88af470b197/users/cee2d65f84924998b642954c037121df/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:19.266844 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cc0414-cb95-4540-90f9-c389c5f4b12a admin admin] RBAC: Authorizing `identity:create_grant(project_id=828263d2c2134eba9781c88af470b197, user_id=cee2d65f84924998b642954c037121df, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:19.269576 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cc0414-cb95-4540-90f9-c389c5f4b12a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:19.284517 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1261/2546] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:19 2022] PUT /identity/v3/projects/828263d2c2134eba9781c88af470b197/users/cee2d65f84924998b642954c037121df/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:19.287919 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2cf21-187c-4553-9d2b-77f8d9d79d61 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:19.299794 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2cf21-187c-4553-9d2b-77f8d9d79d61 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:19.365188 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7398c6a-b221-4da5-8ff7-1492b16fc566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:19.366637 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7398c6a-b221-4da5-8ff7-1492b16fc566 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:19.367127 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7398c6a-b221-4da5-8ff7-1492b16fc566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:19.367486 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7398c6a-b221-4da5-8ff7-1492b16fc566 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:19.368272 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7398c6a-b221-4da5-8ff7-1492b16fc566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:19.370079 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7398c6a-b221-4da5-8ff7-1492b16fc566 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:19.389405 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1286/2547] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:21:19.393910 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cc0414-cb95-4540-90f9-c389c5f4b12a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:19.404698 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cc0414-cb95-4540-90f9-c389c5f4b12a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:19.410037 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b9d1c-38de-4e8a-a56f-587f9eaa0525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:19.410707 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b9d1c-38de-4e8a-a56f-587f9eaa0525 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:19.410997 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b9d1c-38de-4e8a-a56f-587f9eaa0525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:19.411277 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b9d1c-38de-4e8a-a56f-587f9eaa0525 admin admin] PATH_INFO: `/v3/projects/828263d2c2134eba9781c88af470b197/users/cee2d65f84924998b642954c037121df/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:19.413357 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b9d1c-38de-4e8a-a56f-587f9eaa0525 admin admin] RBAC: Authorizing `identity:create_grant(project_id=828263d2c2134eba9781c88af470b197, user_id=cee2d65f84924998b642954c037121df, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:19.415918 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b9d1c-38de-4e8a-a56f-587f9eaa0525 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:19.419152 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:19.428545 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1262/2548] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:19 2022] PUT /identity/v3/projects/828263d2c2134eba9781c88af470b197/users/cee2d65f84924998b642954c037121df/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:19.439524 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840dbd9-3318-4c80-87f2-aa5d3cb201bf None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:19.439807 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840dbd9-3318-4c80-87f2-aa5d3cb201bf None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:19.440089 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840dbd9-3318-4c80-87f2-aa5d3cb201bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:19.472279 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-b840dbd9-3318-4c80-87f2-aa5d3cb201bf None None] MFA Rules not processed for user `cee2d65f84924998b642954c037121df`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:19.473852 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-b840dbd9-3318-4c80-87f2-aa5d3cb201bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:19.550570 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1287/2549] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:19 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:19.561909 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b9d1c-38de-4e8a-a56f-587f9eaa0525 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:19.563601 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-544b9d1c-38de-4e8a-a56f-587f9eaa0525 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:19.621250 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b9d1c-38de-4e8a-a56f-587f9eaa0525 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:19.624865 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd92ee2-79f6-4228-a848-a6c63f184f3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:19.625470 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd92ee2-79f6-4228-a848-a6c63f184f3b service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:19.625749 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd92ee2-79f6-4228-a848-a6c63f184f3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:19.626048 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd92ee2-79f6-4228-a848-a6c63f184f3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:19.634706 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd92ee2-79f6-4228-a848-a6c63f184f3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:19.636217 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd92ee2-79f6-4228-a848-a6c63f184f3b service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:19.645522 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1263/2550] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:19 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:19.814842 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840dbd9-3318-4c80-87f2-aa5d3cb201bf None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:19.827148 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840dbd9-3318-4c80-87f2-aa5d3cb201bf None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:19.830894 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b501b954-84c6-4c65-8663-c612ebf66425 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:19.831511 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b501b954-84c6-4c65-8663-c612ebf66425 service cinder] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:19.831870 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b501b954-84c6-4c65-8663-c612ebf66425 service cinder] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:19.832157 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b501b954-84c6-4c65-8663-c612ebf66425 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:19.862380 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:19.891156 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b501b954-84c6-4c65-8663-c612ebf66425 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:19.892679 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b501b954-84c6-4c65-8663-c612ebf66425 service cinder] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:19.901808 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1288/2551] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:19 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:20.553753 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd92ee2-79f6-4228-a848-a6c63f184f3b service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:20.555176 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-efd92ee2-79f6-4228-a848-a6c63f184f3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:20.620243 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd92ee2-79f6-4228-a848-a6c63f184f3b service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:20.625255 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2360d-cfbe-40a8-bea7-638acf956f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:20.625895 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2360d-cfbe-40a8-bea7-638acf956f5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:20.626179 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2360d-cfbe-40a8-bea7-638acf956f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:20.626462 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2360d-cfbe-40a8-bea7-638acf956f5b admin admin] PATH_INFO: `/v3/users/26a31e77b19c4ba6bf3a003407304b6a` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:20.627621 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f2360d-cfbe-40a8-bea7-638acf956f5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=26a31e77b19c4ba6bf3a003407304b6a)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:20.629590 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f2360d-cfbe-40a8-bea7-638acf956f5b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:20.684493 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-68f2360d-cfbe-40a8-bea7-638acf956f5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26a31e77b19c4ba6bf3a003407304b6a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:20.691500 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-68f2360d-cfbe-40a8-bea7-638acf956f5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26a31e77b19c4ba6bf3a003407304b6a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:20.692183 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-68f2360d-cfbe-40a8-bea7-638acf956f5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26a31e77b19c4ba6bf3a003407304b6a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:20.699835 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-68f2360d-cfbe-40a8-bea7-638acf956f5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26a31e77b19c4ba6bf3a003407304b6a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:20.700441 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-68f2360d-cfbe-40a8-bea7-638acf956f5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26a31e77b19c4ba6bf3a003407304b6a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:20.718780 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1264/2552] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:20 2022] DELETE /identity/v3/users/26a31e77b19c4ba6bf3a003407304b6a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:20.723518 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5fb43-c854-42c6-8ce5-b022fd9f31e4 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:20.723969 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5fb43-c854-42c6-8ce5-b022fd9f31e4 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:20.724067 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5fb43-c854-42c6-8ce5-b022fd9f31e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:20.743215 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-62b5fb43-c854-42c6-8ce5-b022fd9f31e4 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:20.744417 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-62b5fb43-c854-42c6-8ce5-b022fd9f31e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:20.802145 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1289/2553] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:20.812669 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2360d-cfbe-40a8-bea7-638acf956f5b admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:20.814103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-68f2360d-cfbe-40a8-bea7-638acf956f5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:20.834370 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:20.872158 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2360d-cfbe-40a8-bea7-638acf956f5b admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:20.875884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab72d3-c058-45ec-93fb-e01d6f4b9e67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:20.876531 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab72d3-c058-45ec-93fb-e01d6f4b9e67 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:20.876846 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab72d3-c058-45ec-93fb-e01d6f4b9e67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:20.877128 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab72d3-c058-45ec-93fb-e01d6f4b9e67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:20.886638 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ab72d3-c058-45ec-93fb-e01d6f4b9e67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:20.888090 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ab72d3-c058-45ec-93fb-e01d6f4b9e67 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:20.897466 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1265/2554] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:21.164877 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5fb43-c854-42c6-8ce5-b022fd9f31e4 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:21.166286 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-62b5fb43-c854-42c6-8ce5-b022fd9f31e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:21.180541 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5fb43-c854-42c6-8ce5-b022fd9f31e4 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:21.186206 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5182f8d-4398-464e-8e18-8ed97ba3af3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:21.187159 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5182f8d-4398-464e-8e18-8ed97ba3af3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:21.187504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5182f8d-4398-464e-8e18-8ed97ba3af3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:21.187900 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5182f8d-4398-464e-8e18-8ed97ba3af3d admin admin] PATH_INFO: `/v3/projects/03dd0ca268454b3eaf735899752e6d33` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:21.189209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5182f8d-4398-464e-8e18-8ed97ba3af3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=03dd0ca268454b3eaf735899752e6d33)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:21.191126 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5182f8d-4398-464e-8e18-8ed97ba3af3d admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:21.238767 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d5182f8d-4398-464e-8e18-8ed97ba3af3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03dd0ca268454b3eaf735899752e6d33', 'request_id': 'req-d5182f8d-4398-464e-8e18-8ed97ba3af3d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:21.245827 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d5182f8d-4398-464e-8e18-8ed97ba3af3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03dd0ca268454b3eaf735899752e6d33', 'request_id': 'req-d5182f8d-4398-464e-8e18-8ed97ba3af3d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:21.260777 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d5182f8d-4398-464e-8e18-8ed97ba3af3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03dd0ca268454b3eaf735899752e6d33', 'request_id': 'req-d5182f8d-4398-464e-8e18-8ed97ba3af3d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:21.264460 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d5182f8d-4398-464e-8e18-8ed97ba3af3d admin admin] The token cache is being invalidate because project 03dd0ca268454b3eaf735899752e6d33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:21.264793 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d5182f8d-4398-464e-8e18-8ed97ba3af3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:21.265421 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d5182f8d-4398-464e-8e18-8ed97ba3af3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:21.266930 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1290/2555] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:21 2022] DELETE /identity/v3/projects/03dd0ca268454b3eaf735899752e6d33 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:21.271042 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab72d3-c058-45ec-93fb-e01d6f4b9e67 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:21.281082 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab72d3-c058-45ec-93fb-e01d6f4b9e67 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:21.343463 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b24d92-588a-48dd-b376-7cca91671a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:21.344184 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b24d92-588a-48dd-b376-7cca91671a72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:21.344463 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b24d92-588a-48dd-b376-7cca91671a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:21.344805 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b24d92-588a-48dd-b376-7cca91671a72 admin admin] PATH_INFO: `/v3/users/b9155469bda84d2f93f619d1e7c9fa8c` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:21.346306 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b24d92-588a-48dd-b376-7cca91671a72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9155469bda84d2f93f619d1e7c9fa8c)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:21.348397 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b24d92-588a-48dd-b376-7cca91671a72 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:21.358295 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5182f8d-4398-464e-8e18-8ed97ba3af3d admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:21.359979 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-d5182f8d-4398-464e-8e18-8ed97ba3af3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:21.405337 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e0b24d92-588a-48dd-b376-7cca91671a72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9155469bda84d2f93f619d1e7c9fa8c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:21.415118 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e0b24d92-588a-48dd-b376-7cca91671a72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9155469bda84d2f93f619d1e7c9fa8c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:21.415818 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e0b24d92-588a-48dd-b376-7cca91671a72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9155469bda84d2f93f619d1e7c9fa8c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:21.430227 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e0b24d92-588a-48dd-b376-7cca91671a72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9155469bda84d2f93f619d1e7c9fa8c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:21.430864 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e0b24d92-588a-48dd-b376-7cca91671a72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9155469bda84d2f93f619d1e7c9fa8c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:21.444290 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5182f8d-4398-464e-8e18-8ed97ba3af3d admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:21.449210 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fdd1ac-9f3e-4b5c-ac91-f36edcb64cae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:21.450048 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fdd1ac-9f3e-4b5c-ac91-f36edcb64cae service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:21.450403 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fdd1ac-9f3e-4b5c-ac91-f36edcb64cae service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:21.450793 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fdd1ac-9f3e-4b5c-ac91-f36edcb64cae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:21.452833 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-81fdd1ac-9f3e-4b5c-ac91-f36edcb64cae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:21.461145 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1266/2556] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:21 2022] DELETE /identity/v3/users/b9155469bda84d2f93f619d1e7c9fa8c => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:21.468423 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:21.522860 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fdd1ac-9f3e-4b5c-ac91-f36edcb64cae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:21.524255 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fdd1ac-9f3e-4b5c-ac91-f36edcb64cae service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:21.534021 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1291/2557] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:21:21.745290 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b24d92-588a-48dd-b376-7cca91671a72 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:21.748071 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e0b24d92-588a-48dd-b376-7cca91671a72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:21.796430 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:21.836247 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b24d92-588a-48dd-b376-7cca91671a72 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:21.841429 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045cf542-93b7-456a-a784-0d9fa63a1768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:21.842381 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045cf542-93b7-456a-a784-0d9fa63a1768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:21.842893 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045cf542-93b7-456a-a784-0d9fa63a1768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:21.843334 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045cf542-93b7-456a-a784-0d9fa63a1768 admin admin] PATH_INFO: `/v3/projects/47dd527f45dc4c1ba781e99773545627` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:21.844723 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045cf542-93b7-456a-a784-0d9fa63a1768 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47dd527f45dc4c1ba781e99773545627)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:21.847173 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045cf542-93b7-456a-a784-0d9fa63a1768 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:21.907659 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-045cf542-93b7-456a-a784-0d9fa63a1768 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47dd527f45dc4c1ba781e99773545627', 'request_id': 'req-045cf542-93b7-456a-a784-0d9fa63a1768'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:21.917099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-045cf542-93b7-456a-a784-0d9fa63a1768 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47dd527f45dc4c1ba781e99773545627', 'request_id': 'req-045cf542-93b7-456a-a784-0d9fa63a1768'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:21.923964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-045cf542-93b7-456a-a784-0d9fa63a1768 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47dd527f45dc4c1ba781e99773545627', 'request_id': 'req-045cf542-93b7-456a-a784-0d9fa63a1768'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:21.936614 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-045cf542-93b7-456a-a784-0d9fa63a1768 admin admin] The token cache is being invalidate because project 47dd527f45dc4c1ba781e99773545627 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:21.936913 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-045cf542-93b7-456a-a784-0d9fa63a1768 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:21.937572 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-045cf542-93b7-456a-a784-0d9fa63a1768 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:21.939200 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1267/2558] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:21 2022] DELETE /identity/v3/projects/47dd527f45dc4c1ba781e99773545627 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:21.944074 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91357663-91e4-43f1-8f0f-f6f641cb5479 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:21.944578 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91357663-91e4-43f1-8f0f-f6f641cb5479 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:21.944691 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91357663-91e4-43f1-8f0f-f6f641cb5479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:21.967241 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-91357663-91e4-43f1-8f0f-f6f641cb5479 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:21.969482 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-91357663-91e4-43f1-8f0f-f6f641cb5479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:22.065004 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1292/2559] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:22.072646 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3307aa-e1b4-4c16-99c9-bbb73cf47ba0 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:22.073223 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3307aa-e1b4-4c16-99c9-bbb73cf47ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:22.073223 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3307aa-e1b4-4c16-99c9-bbb73cf47ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:22.093585 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ff3307aa-e1b4-4c16-99c9-bbb73cf47ba0 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:22.094816 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ff3307aa-e1b4-4c16-99c9-bbb73cf47ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:22.103760 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1268/2560] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:22.109554 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91357663-91e4-43f1-8f0f-f6f641cb5479 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:22.120661 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91357663-91e4-43f1-8f0f-f6f641cb5479 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:22.125996 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db2d8e-feb2-4b0c-9cdb-18a43152fb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:22.126825 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db2d8e-feb2-4b0c-9cdb-18a43152fb8e admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:22.127123 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db2d8e-feb2-4b0c-9cdb-18a43152fb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:22.127446 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db2d8e-feb2-4b0c-9cdb-18a43152fb8e admin admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:22.128293 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db2d8e-feb2-4b0c-9cdb-18a43152fb8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:21:22.128733 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db2d8e-feb2-4b0c-9cdb-18a43152fb8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:22.130350 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db2d8e-feb2-4b0c-9cdb-18a43152fb8e admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:22.147776 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1293/2561] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:21:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:21:22.150530 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3307aa-e1b4-4c16-99c9-bbb73cf47ba0 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:22.169558 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3307aa-e1b4-4c16-99c9-bbb73cf47ba0 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:22.174613 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686f92e-9964-4d80-95e6-8311ccb5a6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:22.175534 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686f92e-9964-4d80-95e6-8311ccb5a6f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:22.175826 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686f92e-9964-4d80-95e6-8311ccb5a6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:22.176110 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686f92e-9964-4d80-95e6-8311ccb5a6f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:22.177090 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686f92e-9964-4d80-95e6-8311ccb5a6f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:22.179702 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686f92e-9964-4d80-95e6-8311ccb5a6f6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:22.212639 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1269/2562] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:22 2022] POST /identity/v3/projects => generated 369 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:22.216954 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db2d8e-feb2-4b0c-9cdb-18a43152fb8e admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:22.234585 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db2d8e-feb2-4b0c-9cdb-18a43152fb8e admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:22.290191 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e618c-ec46-43d4-a4dd-bf0d25d558e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:22.290743 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e618c-ec46-43d4-a4dd-bf0d25d558e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:22.291024 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e618c-ec46-43d4-a4dd-bf0d25d558e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:22.291296 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e618c-ec46-43d4-a4dd-bf0d25d558e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:22.291963 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734e618c-ec46-43d4-a4dd-bf0d25d558e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:22.293930 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734e618c-ec46-43d4-a4dd-bf0d25d558e8 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:22.317732 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1294/2563] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:22 2022] POST /identity/v3/users => generated 285 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:22.319117 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686f92e-9964-4d80-95e6-8311ccb5a6f6 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:22.330196 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686f92e-9964-4d80-95e6-8311ccb5a6f6 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:22.335441 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312e241-4a68-4aa6-ab2c-7e361225961d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:22.336065 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312e241-4a68-4aa6-ab2c-7e361225961d admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:22.336361 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312e241-4a68-4aa6-ab2c-7e361225961d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:22.336678 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312e241-4a68-4aa6-ab2c-7e361225961d admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:22.337381 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9312e241-4a68-4aa6-ab2c-7e361225961d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:22.339007 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9312e241-4a68-4aa6-ab2c-7e361225961d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:22.349399 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1270/2564] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:22.354336 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e618c-ec46-43d4-a4dd-bf0d25d558e8 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:22.366236 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e618c-ec46-43d4-a4dd-bf0d25d558e8 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:22.370927 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2478f8-15bd-49d1-b22f-f1cbf15343e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:22.371542 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2478f8-15bd-49d1-b22f-f1cbf15343e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:22.371971 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2478f8-15bd-49d1-b22f-f1cbf15343e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:22.372254 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2478f8-15bd-49d1-b22f-f1cbf15343e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:22.372989 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2478f8-15bd-49d1-b22f-f1cbf15343e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:22.374618 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2478f8-15bd-49d1-b22f-f1cbf15343e2 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:22.385586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1295/2565] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:21:22.390007 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312e241-4a68-4aa6-ab2c-7e361225961d admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:22.400571 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312e241-4a68-4aa6-ab2c-7e361225961d admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:22.405500 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979acb97-ed57-4ba0-97af-4ca64f8a78a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:22.406257 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979acb97-ed57-4ba0-97af-4ca64f8a78a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:22.406444 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979acb97-ed57-4ba0-97af-4ca64f8a78a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:22.406721 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979acb97-ed57-4ba0-97af-4ca64f8a78a4 admin admin] PATH_INFO: `/v3/projects/42d00d624bb045d5828d5789a1741d22/users/1e598927d28e4701862d18ebb4153a44/roles/926b9f36c24847c582c84af904250abc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:22.426034 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979acb97-ed57-4ba0-97af-4ca64f8a78a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42d00d624bb045d5828d5789a1741d22, user_id=1e598927d28e4701862d18ebb4153a44, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:22.428905 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979acb97-ed57-4ba0-97af-4ca64f8a78a4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:22.457308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0284ac-ad48-4a5c-96cb-b7465d1aacbd None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:22.457600 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0284ac-ad48-4a5c-96cb-b7465d1aacbd None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:22.457877 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0284ac-ad48-4a5c-96cb-b7465d1aacbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:22.461607 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1271/2566] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:22 2022] PUT /identity/v3/projects/42d00d624bb045d5828d5789a1741d22/users/1e598927d28e4701862d18ebb4153a44/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:22.509352 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-ed0284ac-ad48-4a5c-96cb-b7465d1aacbd None None] MFA Rules not processed for user `1e598927d28e4701862d18ebb4153a44`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:22.510911 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-ed0284ac-ad48-4a5c-96cb-b7465d1aacbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:22.516013 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:22.612305 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1296/2567] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:22 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:22.616972 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979acb97-ed57-4ba0-97af-4ca64f8a78a4 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:22.687333 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979acb97-ed57-4ba0-97af-4ca64f8a78a4 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:22.692439 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c6353-70fa-4685-a33c-dee7dbd2e73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:22.693099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c6353-70fa-4685-a33c-dee7dbd2e73f admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:22.693926 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c6353-70fa-4685-a33c-dee7dbd2e73f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:22.694244 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c6353-70fa-4685-a33c-dee7dbd2e73f admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:22.695050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466c6353-70fa-4685-a33c-dee7dbd2e73f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:22.697124 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466c6353-70fa-4685-a33c-dee7dbd2e73f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:22.709795 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:22.731801 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1272/2568] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:22 2022] POST /identity/v3/projects => generated 369 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:22.736058 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0284ac-ad48-4a5c-96cb-b7465d1aacbd None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:22.748506 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0284ac-ad48-4a5c-96cb-b7465d1aacbd None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:22.812622 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4cbc63-885c-427e-9d40-d147a8008076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:22.813258 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cbc63-885c-427e-9d40-d147a8008076 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:22.813549 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cbc63-885c-427e-9d40-d147a8008076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:22.813831 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cbc63-885c-427e-9d40-d147a8008076 admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:22.814506 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4cbc63-885c-427e-9d40-d147a8008076 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:22.816443 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4cbc63-885c-427e-9d40-d147a8008076 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:22.841519 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1297/2569] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:22 2022] POST /identity/v3/users => generated 291 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:22.845537 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c6353-70fa-4685-a33c-dee7dbd2e73f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:22.856823 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c6353-70fa-4685-a33c-dee7dbd2e73f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:22.861577 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3251ca4e-dbfe-497a-a806-39a047b98b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:22.862200 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251ca4e-dbfe-497a-a806-39a047b98b3c admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:22.862484 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251ca4e-dbfe-497a-a806-39a047b98b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:22.862761 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251ca4e-dbfe-497a-a806-39a047b98b3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:22.863465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3251ca4e-dbfe-497a-a806-39a047b98b3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:22.865114 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3251ca4e-dbfe-497a-a806-39a047b98b3c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:22.879980 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1273/2570] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:22.884500 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4cbc63-885c-427e-9d40-d147a8008076 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:22.894924 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4cbc63-885c-427e-9d40-d147a8008076 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:22.901228 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952c3a7-c8e4-4442-8615-6b0762a7b87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:22.902110 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952c3a7-c8e4-4442-8615-6b0762a7b87f admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:22.902459 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952c3a7-c8e4-4442-8615-6b0762a7b87f admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:22.903001 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952c3a7-c8e4-4442-8615-6b0762a7b87f admin admin] PATH_INFO: `/v3/projects/ffab168784b84452bd3b04fa9b2dacf6/users/ea3a79b035414c52b15b8012336a1b67/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:22.918899 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0952c3a7-c8e4-4442-8615-6b0762a7b87f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffab168784b84452bd3b04fa9b2dacf6, user_id=ea3a79b035414c52b15b8012336a1b67, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:22.921566 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0952c3a7-c8e4-4442-8615-6b0762a7b87f admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:22.935368 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1298/2571] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:22 2022] PUT /identity/v3/projects/ffab168784b84452bd3b04fa9b2dacf6/users/ea3a79b035414c52b15b8012336a1b67/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:22.939568 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3251ca4e-dbfe-497a-a806-39a047b98b3c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:22.950405 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3251ca4e-dbfe-497a-a806-39a047b98b3c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:23.010784 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567dba9a-c6e8-438e-8c96-32b796ffcce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:23.011573 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567dba9a-c6e8-438e-8c96-32b796ffcce4 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:23.011923 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567dba9a-c6e8-438e-8c96-32b796ffcce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:23.012213 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567dba9a-c6e8-438e-8c96-32b796ffcce4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:23.012961 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567dba9a-c6e8-438e-8c96-32b796ffcce4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:23.014743 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567dba9a-c6e8-438e-8c96-32b796ffcce4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:23.024982 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1274/2572] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:23.029155 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952c3a7-c8e4-4442-8615-6b0762a7b87f admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:23.039728 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952c3a7-c8e4-4442-8615-6b0762a7b87f admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:23.044467 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb713ea-55ef-4a2a-95da-3bddc9a395e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:23.045237 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb713ea-55ef-4a2a-95da-3bddc9a395e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:23.045636 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb713ea-55ef-4a2a-95da-3bddc9a395e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:23.046037 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb713ea-55ef-4a2a-95da-3bddc9a395e2 admin admin] PATH_INFO: `/v3/projects/ffab168784b84452bd3b04fa9b2dacf6/users/ea3a79b035414c52b15b8012336a1b67/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:23.048112 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb713ea-55ef-4a2a-95da-3bddc9a395e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffab168784b84452bd3b04fa9b2dacf6, user_id=ea3a79b035414c52b15b8012336a1b67, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:23.050879 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb713ea-55ef-4a2a-95da-3bddc9a395e2 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:23.062062 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1299/2573] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:23 2022] PUT /identity/v3/projects/ffab168784b84452bd3b04fa9b2dacf6/users/ea3a79b035414c52b15b8012336a1b67/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:23.073022 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013472a3-7db5-46af-8f5e-4063b25b8082 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:23.073315 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013472a3-7db5-46af-8f5e-4063b25b8082 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:23.073624 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013472a3-7db5-46af-8f5e-4063b25b8082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:23.104789 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-013472a3-7db5-46af-8f5e-4063b25b8082 None None] MFA Rules not processed for user `ea3a79b035414c52b15b8012336a1b67`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:23.106045 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-013472a3-7db5-46af-8f5e-4063b25b8082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:23.317992 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1275/2574] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:23 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:23.329208 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb713ea-55ef-4a2a-95da-3bddc9a395e2 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:23.331976 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-7bb713ea-55ef-4a2a-95da-3bddc9a395e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:23.393422 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb713ea-55ef-4a2a-95da-3bddc9a395e2 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:23.397651 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1bb74e-2041-4930-a941-50a3b0df3acc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:23.398393 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1bb74e-2041-4930-a941-50a3b0df3acc service cinder] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:23.398795 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1bb74e-2041-4930-a941-50a3b0df3acc service cinder] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:23.399085 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1bb74e-2041-4930-a941-50a3b0df3acc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:23.408380 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1bb74e-2041-4930-a941-50a3b0df3acc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:23.409911 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1bb74e-2041-4930-a941-50a3b0df3acc service cinder] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:23.415612 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:23.421513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1300/2575] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:23 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:24.778751 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013472a3-7db5-46af-8f5e-4063b25b8082 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:24.840771 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013472a3-7db5-46af-8f5e-4063b25b8082 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:24.845263 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668932c-cb51-45d5-91cc-1eb923a326a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:24.845903 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668932c-cb51-45d5-91cc-1eb923a326a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:24.846190 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668932c-cb51-45d5-91cc-1eb923a326a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:24.846470 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668932c-cb51-45d5-91cc-1eb923a326a9 admin admin] PATH_INFO: `/v3/users/1e598927d28e4701862d18ebb4153a44` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:24.847628 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1668932c-cb51-45d5-91cc-1eb923a326a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e598927d28e4701862d18ebb4153a44)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:24.850090 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1668932c-cb51-45d5-91cc-1eb923a326a9 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:24.852351 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:24.897276 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1668932c-cb51-45d5-91cc-1eb923a326a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e598927d28e4701862d18ebb4153a44'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:24.904078 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1668932c-cb51-45d5-91cc-1eb923a326a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e598927d28e4701862d18ebb4153a44'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:24.904764 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1668932c-cb51-45d5-91cc-1eb923a326a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e598927d28e4701862d18ebb4153a44'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:24.912898 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1668932c-cb51-45d5-91cc-1eb923a326a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e598927d28e4701862d18ebb4153a44'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:24.913531 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1668932c-cb51-45d5-91cc-1eb923a326a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e598927d28e4701862d18ebb4153a44'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:24.931967 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1276/2576] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:24 2022] DELETE /identity/v3/users/1e598927d28e4701862d18ebb4153a44 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:24.936665 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb14491-920e-42eb-808f-a5209d88da75 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:24.937402 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb14491-920e-42eb-808f-a5209d88da75 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:24.937402 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb14491-920e-42eb-808f-a5209d88da75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:24.956101 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-2cb14491-920e-42eb-808f-a5209d88da75 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:24.957655 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-2cb14491-920e-42eb-808f-a5209d88da75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:25.019228 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1301/2577] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:25.028964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668932c-cb51-45d5-91cc-1eb923a326a9 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:25.030375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1668932c-cb51-45d5-91cc-1eb923a326a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:25.088453 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668932c-cb51-45d5-91cc-1eb923a326a9 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:25.092178 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caddd46-4e8f-47b0-8e8d-25dcd81fd8d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:25.092776 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caddd46-4e8f-47b0-8e8d-25dcd81fd8d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:25.093059 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caddd46-4e8f-47b0-8e8d-25dcd81fd8d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:25.093337 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caddd46-4e8f-47b0-8e8d-25dcd81fd8d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:25.103185 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caddd46-4e8f-47b0-8e8d-25dcd81fd8d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:25.104642 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caddd46-4e8f-47b0-8e8d-25dcd81fd8d6 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:25.114238 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1277/2578] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:25.376001 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb14491-920e-42eb-808f-a5209d88da75 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:25.377464 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-2cb14491-920e-42eb-808f-a5209d88da75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:25.390146 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb14491-920e-42eb-808f-a5209d88da75 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:25.395146 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2746d-84ca-4e42-a27f-5e039fbb11c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:25.395840 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2746d-84ca-4e42-a27f-5e039fbb11c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:25.396131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2746d-84ca-4e42-a27f-5e039fbb11c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:25.396986 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2746d-84ca-4e42-a27f-5e039fbb11c2 admin admin] PATH_INFO: `/v3/projects/42d00d624bb045d5828d5789a1741d22` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:25.398884 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a2746d-84ca-4e42-a27f-5e039fbb11c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42d00d624bb045d5828d5789a1741d22)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:25.401247 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a2746d-84ca-4e42-a27f-5e039fbb11c2 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:25.453360 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-06a2746d-84ca-4e42-a27f-5e039fbb11c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42d00d624bb045d5828d5789a1741d22', 'request_id': 'req-06a2746d-84ca-4e42-a27f-5e039fbb11c2'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:25.461137 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-06a2746d-84ca-4e42-a27f-5e039fbb11c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42d00d624bb045d5828d5789a1741d22', 'request_id': 'req-06a2746d-84ca-4e42-a27f-5e039fbb11c2'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:25.467100 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-06a2746d-84ca-4e42-a27f-5e039fbb11c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42d00d624bb045d5828d5789a1741d22', 'request_id': 'req-06a2746d-84ca-4e42-a27f-5e039fbb11c2'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:25.481473 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-06a2746d-84ca-4e42-a27f-5e039fbb11c2 admin admin] The token cache is being invalidate because project 42d00d624bb045d5828d5789a1741d22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:25.481766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-06a2746d-84ca-4e42-a27f-5e039fbb11c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:25.482415 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-06a2746d-84ca-4e42-a27f-5e039fbb11c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:25.483957 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1302/2579] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:25 2022] DELETE /identity/v3/projects/42d00d624bb045d5828d5789a1741d22 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:25.487875 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caddd46-4e8f-47b0-8e8d-25dcd81fd8d6 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:25.497490 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caddd46-4e8f-47b0-8e8d-25dcd81fd8d6 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:25.552414 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602cdb5-5987-42e9-91f5-8575b6aaccb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:25.553079 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602cdb5-5987-42e9-91f5-8575b6aaccb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:25.553362 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602cdb5-5987-42e9-91f5-8575b6aaccb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:25.553641 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602cdb5-5987-42e9-91f5-8575b6aaccb8 admin admin] PATH_INFO: `/v3/users/ea3a79b035414c52b15b8012336a1b67` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:25.554792 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d602cdb5-5987-42e9-91f5-8575b6aaccb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea3a79b035414c52b15b8012336a1b67)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:25.556718 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d602cdb5-5987-42e9-91f5-8575b6aaccb8 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:25.586023 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:25.606504 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d602cdb5-5987-42e9-91f5-8575b6aaccb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea3a79b035414c52b15b8012336a1b67'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:25.613506 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d602cdb5-5987-42e9-91f5-8575b6aaccb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea3a79b035414c52b15b8012336a1b67'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:25.614170 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d602cdb5-5987-42e9-91f5-8575b6aaccb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea3a79b035414c52b15b8012336a1b67'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:25.621846 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d602cdb5-5987-42e9-91f5-8575b6aaccb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea3a79b035414c52b15b8012336a1b67'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:25.622716 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d602cdb5-5987-42e9-91f5-8575b6aaccb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea3a79b035414c52b15b8012336a1b67'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:25.640390 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1278/2580] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:25 2022] DELETE /identity/v3/users/ea3a79b035414c52b15b8012336a1b67 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:25.915427 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2746d-84ca-4e42-a27f-5e039fbb11c2 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:25.916990 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-06a2746d-84ca-4e42-a27f-5e039fbb11c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:25.930247 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2746d-84ca-4e42-a27f-5e039fbb11c2 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:25.987699 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410bdaf4-11a1-41ca-ba2b-fa01107f6203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:25.988351 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410bdaf4-11a1-41ca-ba2b-fa01107f6203 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:25.988672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410bdaf4-11a1-41ca-ba2b-fa01107f6203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:25.988954 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410bdaf4-11a1-41ca-ba2b-fa01107f6203 admin admin] PATH_INFO: `/v3/projects/ffab168784b84452bd3b04fa9b2dacf6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:25.990650 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410bdaf4-11a1-41ca-ba2b-fa01107f6203 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffab168784b84452bd3b04fa9b2dacf6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:25.992586 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410bdaf4-11a1-41ca-ba2b-fa01107f6203 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:26.023069 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:26.041011 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-410bdaf4-11a1-41ca-ba2b-fa01107f6203 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffab168784b84452bd3b04fa9b2dacf6', 'request_id': 'req-410bdaf4-11a1-41ca-ba2b-fa01107f6203'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:26.047176 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-410bdaf4-11a1-41ca-ba2b-fa01107f6203 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffab168784b84452bd3b04fa9b2dacf6', 'request_id': 'req-410bdaf4-11a1-41ca-ba2b-fa01107f6203'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:26.053263 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-410bdaf4-11a1-41ca-ba2b-fa01107f6203 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffab168784b84452bd3b04fa9b2dacf6', 'request_id': 'req-410bdaf4-11a1-41ca-ba2b-fa01107f6203'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:26.066068 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-410bdaf4-11a1-41ca-ba2b-fa01107f6203 admin admin] The token cache is being invalidate because project ffab168784b84452bd3b04fa9b2dacf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:26.066360 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-410bdaf4-11a1-41ca-ba2b-fa01107f6203 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:26.067001 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-410bdaf4-11a1-41ca-ba2b-fa01107f6203 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:26.069349 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1303/2581] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:25 2022] DELETE /identity/v3/projects/ffab168784b84452bd3b04fa9b2dacf6 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:26.074203 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75e1b1-b4ab-4818-971e-8fd3d004406f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:26.074686 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75e1b1-b4ab-4818-971e-8fd3d004406f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:26.074787 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75e1b1-b4ab-4818-971e-8fd3d004406f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:26.093690 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ea75e1b1-b4ab-4818-971e-8fd3d004406f None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:26.094888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ea75e1b1-b4ab-4818-971e-8fd3d004406f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:26.152874 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1279/2582] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:26.161738 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92299cc6-a9f3-47a6-9387-f55c5f85aab0 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:26.162106 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92299cc6-a9f3-47a6-9387-f55c5f85aab0 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:26.162394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92299cc6-a9f3-47a6-9387-f55c5f85aab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:26.184361 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-92299cc6-a9f3-47a6-9387-f55c5f85aab0 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:26.185598 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-92299cc6-a9f3-47a6-9387-f55c5f85aab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:26.194271 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1304/2583] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:26.198571 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea75e1b1-b4ab-4818-971e-8fd3d004406f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:26.208787 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea75e1b1-b4ab-4818-971e-8fd3d004406f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:26.213475 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f0e2a-b09e-4761-9f86-4b6a030de308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:26.214097 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f0e2a-b09e-4761-9f86-4b6a030de308 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:26.214535 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f0e2a-b09e-4761-9f86-4b6a030de308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:26.214821 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f0e2a-b09e-4761-9f86-4b6a030de308 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:26.215717 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f0e2a-b09e-4761-9f86-4b6a030de308 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:21:26.215876 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f0e2a-b09e-4761-9f86-4b6a030de308 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:26.217507 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f0e2a-b09e-4761-9f86-4b6a030de308 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:26.231411 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1280/2584] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:21:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:21:26.235843 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92299cc6-a9f3-47a6-9387-f55c5f85aab0 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:26.246109 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92299cc6-a9f3-47a6-9387-f55c5f85aab0 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:26.250875 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8c65d-fcbf-4e4a-bb58-70b181d0e946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:26.251568 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8c65d-fcbf-4e4a-bb58-70b181d0e946 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:26.251938 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8c65d-fcbf-4e4a-bb58-70b181d0e946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:26.252279 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8c65d-fcbf-4e4a-bb58-70b181d0e946 admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:26.253067 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca8c65d-fcbf-4e4a-bb58-70b181d0e946 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:26.255029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca8c65d-fcbf-4e4a-bb58-70b181d0e946 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:26.284699 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1305/2585] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:26 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:26.345381 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f0e2a-b09e-4761-9f86-4b6a030de308 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:26.355884 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f0e2a-b09e-4761-9f86-4b6a030de308 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:26.410656 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6564d8e-17fd-4cff-96af-8aa4953fbf3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:26.411272 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6564d8e-17fd-4cff-96af-8aa4953fbf3d admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:26.411608 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6564d8e-17fd-4cff-96af-8aa4953fbf3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:26.412349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6564d8e-17fd-4cff-96af-8aa4953fbf3d admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:26.413043 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6564d8e-17fd-4cff-96af-8aa4953fbf3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:26.414936 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6564d8e-17fd-4cff-96af-8aa4953fbf3d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:26.435803 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1281/2586] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:26 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:26.440004 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8c65d-fcbf-4e4a-bb58-70b181d0e946 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:26.450744 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8c65d-fcbf-4e4a-bb58-70b181d0e946 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:26.457216 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e3a13-2b09-4894-9cfe-c31e8433aa94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:26.457483 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e3a13-2b09-4894-9cfe-c31e8433aa94 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:26.457697 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e3a13-2b09-4894-9cfe-c31e8433aa94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:26.458142 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e3a13-2b09-4894-9cfe-c31e8433aa94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:26.460397 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9e3a13-2b09-4894-9cfe-c31e8433aa94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:26.462580 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9e3a13-2b09-4894-9cfe-c31e8433aa94 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:26.476407 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1306/2587] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:26.480988 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6564d8e-17fd-4cff-96af-8aa4953fbf3d admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:26.491471 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6564d8e-17fd-4cff-96af-8aa4953fbf3d admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:26.496359 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7f50e-8485-4dbe-809e-29a281bbf5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:26.497004 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7f50e-8485-4dbe-809e-29a281bbf5eb admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:26.497292 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7f50e-8485-4dbe-809e-29a281bbf5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:26.497578 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7f50e-8485-4dbe-809e-29a281bbf5eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:26.498268 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa7f50e-8485-4dbe-809e-29a281bbf5eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:26.499849 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa7f50e-8485-4dbe-809e-29a281bbf5eb admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:26.511565 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1282/2588] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:26.516160 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e3a13-2b09-4894-9cfe-c31e8433aa94 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:26.527667 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e3a13-2b09-4894-9cfe-c31e8433aa94 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:26.533940 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea2aad-c9a7-4157-a0cb-3b6e26dad208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:26.534848 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea2aad-c9a7-4157-a0cb-3b6e26dad208 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:26.535157 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea2aad-c9a7-4157-a0cb-3b6e26dad208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:26.535464 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea2aad-c9a7-4157-a0cb-3b6e26dad208 admin admin] PATH_INFO: `/v3/projects/eff6ded2c36846579af2f59ea9bfff8a/users/45782c948da04daab84db1d8de609daf/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:26.553758 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aea2aad-c9a7-4157-a0cb-3b6e26dad208 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eff6ded2c36846579af2f59ea9bfff8a, user_id=45782c948da04daab84db1d8de609daf, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:26.556609 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aea2aad-c9a7-4157-a0cb-3b6e26dad208 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:26.569946 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1307/2589] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:26 2022] PUT /identity/v3/projects/eff6ded2c36846579af2f59ea9bfff8a/users/45782c948da04daab84db1d8de609daf/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:26.580025 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a45146-09dc-424b-a8eb-ac1435a7d377 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:26.580323 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a45146-09dc-424b-a8eb-ac1435a7d377 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:26.580652 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a45146-09dc-424b-a8eb-ac1435a7d377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:26.606591 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:26.619662 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-06a45146-09dc-424b-a8eb-ac1435a7d377 None None] MFA Rules not processed for user `45782c948da04daab84db1d8de609daf`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:26.621373 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-06a45146-09dc-424b-a8eb-ac1435a7d377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:26.701404 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1283/2590] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:26.704903 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea2aad-c9a7-4157-a0cb-3b6e26dad208 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:26.777453 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea2aad-c9a7-4157-a0cb-3b6e26dad208 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:26.782334 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefcc91e-40e8-4c5d-801a-65bf236168de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:26.783007 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefcc91e-40e8-4c5d-801a-65bf236168de admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:26.783297 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefcc91e-40e8-4c5d-801a-65bf236168de admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:26.783579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefcc91e-40e8-4c5d-801a-65bf236168de admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:26.784292 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefcc91e-40e8-4c5d-801a-65bf236168de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:26.786255 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefcc91e-40e8-4c5d-801a-65bf236168de admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:26.826779 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1308/2591] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:26 2022] POST /identity/v3/projects => generated 387 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:26.834078 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a45146-09dc-424b-a8eb-ac1435a7d377 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:26.854455 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a45146-09dc-424b-a8eb-ac1435a7d377 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:26.920529 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae67cb-bbe0-4368-8c7c-f8761391a863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:26.921491 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae67cb-bbe0-4368-8c7c-f8761391a863 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:26.921950 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae67cb-bbe0-4368-8c7c-f8761391a863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:26.922262 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae67cb-bbe0-4368-8c7c-f8761391a863 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:26.922942 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae67cb-bbe0-4368-8c7c-f8761391a863 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:26.925180 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae67cb-bbe0-4368-8c7c-f8761391a863 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:26.956648 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1284/2592] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:26 2022] POST /identity/v3/users => generated 294 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:26.956854 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefcc91e-40e8-4c5d-801a-65bf236168de admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:26.969486 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefcc91e-40e8-4c5d-801a-65bf236168de admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:26.975013 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc040c8b-9f97-4999-9dd7-11bc32cd6b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:26.975746 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc040c8b-9f97-4999-9dd7-11bc32cd6b24 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:26.976107 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc040c8b-9f97-4999-9dd7-11bc32cd6b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:26.976511 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc040c8b-9f97-4999-9dd7-11bc32cd6b24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:26.977359 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc040c8b-9f97-4999-9dd7-11bc32cd6b24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:26.979075 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc040c8b-9f97-4999-9dd7-11bc32cd6b24 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:26.996513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1309/2593] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:21:27.003066 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae67cb-bbe0-4368-8c7c-f8761391a863 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:27.015622 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae67cb-bbe0-4368-8c7c-f8761391a863 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:27.020856 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da30fac3-e064-4372-998e-aca6f6964b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:27.022030 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30fac3-e064-4372-998e-aca6f6964b82 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:27.022441 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30fac3-e064-4372-998e-aca6f6964b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:27.022690 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30fac3-e064-4372-998e-aca6f6964b82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:27.023453 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da30fac3-e064-4372-998e-aca6f6964b82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:27.025137 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da30fac3-e064-4372-998e-aca6f6964b82 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:27.040157 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1285/2594] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:27.045397 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc040c8b-9f97-4999-9dd7-11bc32cd6b24 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:27.058306 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc040c8b-9f97-4999-9dd7-11bc32cd6b24 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:27.063761 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de2422-9a6a-4698-88e3-1035e2ef490a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:27.064667 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de2422-9a6a-4698-88e3-1035e2ef490a admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:27.065025 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de2422-9a6a-4698-88e3-1035e2ef490a admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:27.065394 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de2422-9a6a-4698-88e3-1035e2ef490a admin admin] PATH_INFO: `/v3/projects/9bdbd8cad10f4e04b7493f39235c815e/users/36a7c310b6534ceca2d636aa722b977e/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:27.089090 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51de2422-9a6a-4698-88e3-1035e2ef490a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bdbd8cad10f4e04b7493f39235c815e, user_id=36a7c310b6534ceca2d636aa722b977e, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:27.092938 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51de2422-9a6a-4698-88e3-1035e2ef490a admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:27.103545 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:27.112578 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1310/2595] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:27 2022] PUT /identity/v3/projects/9bdbd8cad10f4e04b7493f39235c815e/users/36a7c310b6534ceca2d636aa722b977e/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:27.123876 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa58ef-85a2-4c73-bf70-ae2604e2fd5b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:27.124957 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa58ef-85a2-4c73-bf70-ae2604e2fd5b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:27.124957 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa58ef-85a2-4c73-bf70-ae2604e2fd5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:27.157769 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a6aa58ef-85a2-4c73-bf70-ae2604e2fd5b None None] MFA Rules not processed for user `36a7c310b6534ceca2d636aa722b977e`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:27.159171 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a6aa58ef-85a2-4c73-bf70-ae2604e2fd5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:27.223275 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1286/2596] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:27.228183 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de2422-9a6a-4698-88e3-1035e2ef490a admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:27.289432 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de2422-9a6a-4698-88e3-1035e2ef490a admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:27.294357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c209339-d1b4-4241-9d11-d5c555a75be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:27.294975 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c209339-d1b4-4241-9d11-d5c555a75be8 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:27.295253 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c209339-d1b4-4241-9d11-d5c555a75be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:27.295528 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c209339-d1b4-4241-9d11-d5c555a75be8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:27.296183 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c209339-d1b4-4241-9d11-d5c555a75be8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:27.298096 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c209339-d1b4-4241-9d11-d5c555a75be8 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:27.325699 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1311/2597] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:27 2022] POST /identity/v3/projects => generated 387 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:27.329815 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa58ef-85a2-4c73-bf70-ae2604e2fd5b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:27.340674 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa58ef-85a2-4c73-bf70-ae2604e2fd5b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:27.344591 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:27.402402 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197d8bd-04d3-4b01-acb3-2f9daec78223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:27.403030 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197d8bd-04d3-4b01-acb3-2f9daec78223 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:27.403317 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197d8bd-04d3-4b01-acb3-2f9daec78223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:27.403600 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197d8bd-04d3-4b01-acb3-2f9daec78223 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:27.404278 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5197d8bd-04d3-4b01-acb3-2f9daec78223 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:27.406235 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5197d8bd-04d3-4b01-acb3-2f9daec78223 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:27.429205 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1287/2598] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:27 2022] POST /identity/v3/users => generated 300 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:27.433207 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c209339-d1b4-4241-9d11-d5c555a75be8 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:27.444211 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c209339-d1b4-4241-9d11-d5c555a75be8 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:27.450676 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f30f6-275b-4346-8ad2-049c8afb2060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:27.451454 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f30f6-275b-4346-8ad2-049c8afb2060 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:27.451855 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f30f6-275b-4346-8ad2-049c8afb2060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:27.452232 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f30f6-275b-4346-8ad2-049c8afb2060 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:27.453173 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4f30f6-275b-4346-8ad2-049c8afb2060 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:27.454928 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4f30f6-275b-4346-8ad2-049c8afb2060 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:27.466670 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1312/2599] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:27.471333 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197d8bd-04d3-4b01-acb3-2f9daec78223 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:27.482576 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197d8bd-04d3-4b01-acb3-2f9daec78223 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:27.487390 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2a8ab-5922-42de-872c-cf8da817932a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:27.488071 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2a8ab-5922-42de-872c-cf8da817932a admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:27.488356 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2a8ab-5922-42de-872c-cf8da817932a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:27.488678 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2a8ab-5922-42de-872c-cf8da817932a admin admin] PATH_INFO: `/v3/projects/10c3c2edbbd2452bb94379fb9f28f871/users/8faf04bf3b10453a975740b2aa080daf/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:27.504068 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b2a8ab-5922-42de-872c-cf8da817932a admin admin] RBAC: Authorizing `identity:create_grant(project_id=10c3c2edbbd2452bb94379fb9f28f871, user_id=8faf04bf3b10453a975740b2aa080daf, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:27.506829 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b2a8ab-5922-42de-872c-cf8da817932a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:27.521805 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1288/2600] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:27 2022] PUT /identity/v3/projects/10c3c2edbbd2452bb94379fb9f28f871/users/8faf04bf3b10453a975740b2aa080daf/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:27.526043 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f30f6-275b-4346-8ad2-049c8afb2060 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:27.537464 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f30f6-275b-4346-8ad2-049c8afb2060 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:27.609277 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d80ae2-ee88-4d25-a6db-3271b8cc4220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:27.610042 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d80ae2-ee88-4d25-a6db-3271b8cc4220 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:27.610523 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d80ae2-ee88-4d25-a6db-3271b8cc4220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:27.610901 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d80ae2-ee88-4d25-a6db-3271b8cc4220 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:27.611802 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d80ae2-ee88-4d25-a6db-3271b8cc4220 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:27.613585 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d80ae2-ee88-4d25-a6db-3271b8cc4220 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:27.626596 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1313/2601] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:27.630850 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2a8ab-5922-42de-872c-cf8da817932a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:27.642592 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2a8ab-5922-42de-872c-cf8da817932a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:27.648336 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b753a-22ad-4106-a23e-e504b5d0d6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:27.649357 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b753a-22ad-4106-a23e-e504b5d0d6e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:27.649717 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b753a-22ad-4106-a23e-e504b5d0d6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:27.650082 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b753a-22ad-4106-a23e-e504b5d0d6e8 admin admin] PATH_INFO: `/v3/projects/10c3c2edbbd2452bb94379fb9f28f871/users/8faf04bf3b10453a975740b2aa080daf/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:27.652838 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b753a-22ad-4106-a23e-e504b5d0d6e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10c3c2edbbd2452bb94379fb9f28f871, user_id=8faf04bf3b10453a975740b2aa080daf, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:27.656331 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b753a-22ad-4106-a23e-e504b5d0d6e8 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:27.676531 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1289/2602] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:27 2022] PUT /identity/v3/projects/10c3c2edbbd2452bb94379fb9f28f871/users/8faf04bf3b10453a975740b2aa080daf/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:27.688750 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382d18f-0b73-4fc9-868c-8e665d8bebb1 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:27.689058 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382d18f-0b73-4fc9-868c-8e665d8bebb1 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:27.689361 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382d18f-0b73-4fc9-868c-8e665d8bebb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:27.727413 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-c382d18f-0b73-4fc9-868c-8e665d8bebb1 None None] MFA Rules not processed for user `8faf04bf3b10453a975740b2aa080daf`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:27.728895 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-c382d18f-0b73-4fc9-868c-8e665d8bebb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:27.812641 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:27.823760 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1314/2603] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:27 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:27.834800 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b753a-22ad-4106-a23e-e504b5d0d6e8 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:27.836593 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-956b753a-22ad-4106-a23e-e504b5d0d6e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:27.887706 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c382d18f-0b73-4fc9-868c-8e665d8bebb1 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:27.889165 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-c382d18f-0b73-4fc9-868c-8e665d8bebb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:27.906603 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b753a-22ad-4106-a23e-e504b5d0d6e8 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:27.910351 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499dac16-48cb-48ac-8181-308a8c114060 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:27.910995 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499dac16-48cb-48ac-8181-308a8c114060 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:27.911276 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499dac16-48cb-48ac-8181-308a8c114060 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:27.911545 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499dac16-48cb-48ac-8181-308a8c114060 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:27.921124 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499dac16-48cb-48ac-8181-308a8c114060 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:27.922556 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499dac16-48cb-48ac-8181-308a8c114060 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:27.933377 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1290/2604] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:27 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:27.978936 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c382d18f-0b73-4fc9-868c-8e665d8bebb1 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:27.988474 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107039bb-5217-4a6a-b7cf-a5efc6beb3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:27.989278 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107039bb-5217-4a6a-b7cf-a5efc6beb3ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:27.989440 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107039bb-5217-4a6a-b7cf-a5efc6beb3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:27.989795 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107039bb-5217-4a6a-b7cf-a5efc6beb3ae admin admin] PATH_INFO: `/v3/users/50ef2b0200604e9995c89f757521825a` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:27.991163 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107039bb-5217-4a6a-b7cf-a5efc6beb3ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=50ef2b0200604e9995c89f757521825a)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:27.993512 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107039bb-5217-4a6a-b7cf-a5efc6beb3ae admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:28.069752 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-107039bb-5217-4a6a-b7cf-a5efc6beb3ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50ef2b0200604e9995c89f757521825a'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:28.071147 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-107039bb-5217-4a6a-b7cf-a5efc6beb3ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50ef2b0200604e9995c89f757521825a'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:28.087172 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-107039bb-5217-4a6a-b7cf-a5efc6beb3ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50ef2b0200604e9995c89f757521825a'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:28.097070 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-107039bb-5217-4a6a-b7cf-a5efc6beb3ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50ef2b0200604e9995c89f757521825a'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:28.097950 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-107039bb-5217-4a6a-b7cf-a5efc6beb3ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50ef2b0200604e9995c89f757521825a'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:28.115371 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1315/2605] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:27 2022] DELETE /identity/v3/users/50ef2b0200604e9995c89f757521825a => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:28.121918 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb6573-3844-4ffe-9e82-a7ae66323c79 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:28.122208 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb6573-3844-4ffe-9e82-a7ae66323c79 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:28.122487 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb6573-3844-4ffe-9e82-a7ae66323c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:28.142784 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-17bb6573-3844-4ffe-9e82-a7ae66323c79 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:28.144056 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-17bb6573-3844-4ffe-9e82-a7ae66323c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:28.245686 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1291/2606] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:21:28.253880 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107039bb-5217-4a6a-b7cf-a5efc6beb3ae admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:28.255443 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-107039bb-5217-4a6a-b7cf-a5efc6beb3ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:28.327979 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107039bb-5217-4a6a-b7cf-a5efc6beb3ae admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:28.332822 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a39144-1bda-4dcc-94de-0579a280bfc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:28.334544 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a39144-1bda-4dcc-94de-0579a280bfc6 service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:28.335231 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a39144-1bda-4dcc-94de-0579a280bfc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:28.335516 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a39144-1bda-4dcc-94de-0579a280bfc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:28.348737 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a39144-1bda-4dcc-94de-0579a280bfc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:28.353506 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a39144-1bda-4dcc-94de-0579a280bfc6 service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:28.367492 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1316/2607] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:28.607569 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb6573-3844-4ffe-9e82-a7ae66323c79 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:28.609227 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-17bb6573-3844-4ffe-9e82-a7ae66323c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:28.621527 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:28.726152 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb6573-3844-4ffe-9e82-a7ae66323c79 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:28.730928 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e27a39b-6d56-411d-97b6-c88f7cb2187d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:28.731581 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27a39b-6d56-411d-97b6-c88f7cb2187d service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:28.731845 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27a39b-6d56-411d-97b6-c88f7cb2187d service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:28.732128 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27a39b-6d56-411d-97b6-c88f7cb2187d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:28.735700 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0e27a39b-6d56-411d-97b6-c88f7cb2187d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:28.774708 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a39144-1bda-4dcc-94de-0579a280bfc6 service neutron] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:28.778921 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-b3a39144-1bda-4dcc-94de-0579a280bfc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:28.805728 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a39144-1bda-4dcc-94de-0579a280bfc6 service neutron] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:28.819694 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3a6443-cc02-43a7-8b43-1707df5ffed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:28.820359 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a6443-cc02-43a7-8b43-1707df5ffed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:28.820862 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a6443-cc02-43a7-8b43-1707df5ffed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:28.821172 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a6443-cc02-43a7-8b43-1707df5ffed7 admin admin] PATH_INFO: `/v3/projects/efe741d706974bb0b475d7450a87fe26` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:28.822361 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3a6443-cc02-43a7-8b43-1707df5ffed7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efe741d706974bb0b475d7450a87fe26)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:28.824400 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3a6443-cc02-43a7-8b43-1707df5ffed7 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:28.838824 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:28.845223 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e27a39b-6d56-411d-97b6-c88f7cb2187d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:28.846793 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e27a39b-6d56-411d-97b6-c88f7cb2187d service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:28.858527 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1292/2608] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:28 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:28.898910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-cf3a6443-cc02-43a7-8b43-1707df5ffed7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efe741d706974bb0b475d7450a87fe26', 'request_id': 'req-cf3a6443-cc02-43a7-8b43-1707df5ffed7'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:28.908426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-cf3a6443-cc02-43a7-8b43-1707df5ffed7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efe741d706974bb0b475d7450a87fe26', 'request_id': 'req-cf3a6443-cc02-43a7-8b43-1707df5ffed7'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:28.915214 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-cf3a6443-cc02-43a7-8b43-1707df5ffed7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efe741d706974bb0b475d7450a87fe26', 'request_id': 'req-cf3a6443-cc02-43a7-8b43-1707df5ffed7'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:28.932211 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-cf3a6443-cc02-43a7-8b43-1707df5ffed7 admin admin] The token cache is being invalidate because project efe741d706974bb0b475d7450a87fe26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:28.932528 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-cf3a6443-cc02-43a7-8b43-1707df5ffed7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:28.933181 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-cf3a6443-cc02-43a7-8b43-1707df5ffed7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:28.935390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1317/2609] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:28 2022] DELETE /identity/v3/projects/efe741d706974bb0b475d7450a87fe26 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:28.939708 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e27a39b-6d56-411d-97b6-c88f7cb2187d service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:28.952259 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e27a39b-6d56-411d-97b6-c88f7cb2187d service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:29.019175 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad678f9d-8012-4b18-a86f-ec1903eefe4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:29.019826 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad678f9d-8012-4b18-a86f-ec1903eefe4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:29.020146 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad678f9d-8012-4b18-a86f-ec1903eefe4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:29.020382 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad678f9d-8012-4b18-a86f-ec1903eefe4a admin admin] PATH_INFO: `/v3/users/9bf3ebe8ea3943638f7db3ef1966bb7e` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:29.021751 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad678f9d-8012-4b18-a86f-ec1903eefe4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bf3ebe8ea3943638f7db3ef1966bb7e)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:29.023723 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad678f9d-8012-4b18-a86f-ec1903eefe4a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:29.071699 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ad678f9d-8012-4b18-a86f-ec1903eefe4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bf3ebe8ea3943638f7db3ef1966bb7e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:29.079466 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ad678f9d-8012-4b18-a86f-ec1903eefe4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bf3ebe8ea3943638f7db3ef1966bb7e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:29.080179 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ad678f9d-8012-4b18-a86f-ec1903eefe4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bf3ebe8ea3943638f7db3ef1966bb7e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:29.083241 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:29.088617 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ad678f9d-8012-4b18-a86f-ec1903eefe4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bf3ebe8ea3943638f7db3ef1966bb7e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:29.089757 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ad678f9d-8012-4b18-a86f-ec1903eefe4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bf3ebe8ea3943638f7db3ef1966bb7e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:29.109462 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1293/2610] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:28 2022] DELETE /identity/v3/users/9bf3ebe8ea3943638f7db3ef1966bb7e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:29.402490 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3a6443-cc02-43a7-8b43-1707df5ffed7 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:29.404173 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-cf3a6443-cc02-43a7-8b43-1707df5ffed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:29.426000 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3a6443-cc02-43a7-8b43-1707df5ffed7 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:29.494972 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d830b-1016-4d53-b901-77e8dfed362d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:29.495712 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d830b-1016-4d53-b901-77e8dfed362d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:29.495999 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d830b-1016-4d53-b901-77e8dfed362d admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:29.496304 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d830b-1016-4d53-b901-77e8dfed362d admin admin] PATH_INFO: `/v3/projects/ef9a0bb54d6249468a6788594e930b92` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:29.497618 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1d830b-1016-4d53-b901-77e8dfed362d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef9a0bb54d6249468a6788594e930b92)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:29.500160 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1d830b-1016-4d53-b901-77e8dfed362d admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:29.553895 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-9c1d830b-1016-4d53-b901-77e8dfed362d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef9a0bb54d6249468a6788594e930b92', 'request_id': 'req-9c1d830b-1016-4d53-b901-77e8dfed362d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:29.561525 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-9c1d830b-1016-4d53-b901-77e8dfed362d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef9a0bb54d6249468a6788594e930b92', 'request_id': 'req-9c1d830b-1016-4d53-b901-77e8dfed362d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:29.569189 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-9c1d830b-1016-4d53-b901-77e8dfed362d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef9a0bb54d6249468a6788594e930b92', 'request_id': 'req-9c1d830b-1016-4d53-b901-77e8dfed362d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:29.588646 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-9c1d830b-1016-4d53-b901-77e8dfed362d admin admin] The token cache is being invalidate because project ef9a0bb54d6249468a6788594e930b92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:29.588980 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-9c1d830b-1016-4d53-b901-77e8dfed362d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:29.589690 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-9c1d830b-1016-4d53-b901-77e8dfed362d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:29.591411 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1318/2611] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:29 2022] DELETE /identity/v3/projects/ef9a0bb54d6249468a6788594e930b92 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:29.594356 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad678f9d-8012-4b18-a86f-ec1903eefe4a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:29.595915 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ad678f9d-8012-4b18-a86f-ec1903eefe4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:29.597467 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae49edf7-ac2f-423d-99d4-ec33f85d82f2 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:29.597759 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae49edf7-ac2f-423d-99d4-ec33f85d82f2 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:29.598045 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae49edf7-ac2f-423d-99d4-ec33f85d82f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:29.610147 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad678f9d-8012-4b18-a86f-ec1903eefe4a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:29.618174 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-ae49edf7-ac2f-423d-99d4-ec33f85d82f2 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:29.619390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-ae49edf7-ac2f-423d-99d4-ec33f85d82f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:29.645564 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:29.669605 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d859a-956e-49a4-9f7a-b8a5a8c50635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:29.670524 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d859a-956e-49a4-9f7a-b8a5a8c50635 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:29.670810 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d859a-956e-49a4-9f7a-b8a5a8c50635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:29.671089 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d859a-956e-49a4-9f7a-b8a5a8c50635 admin admin] PATH_INFO: `/v3/users/c96f29dc500445ffbd5a2f2af965e16e` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:29.672243 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459d859a-956e-49a4-9f7a-b8a5a8c50635 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c96f29dc500445ffbd5a2f2af965e16e)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:29.674249 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459d859a-956e-49a4-9f7a-b8a5a8c50635 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:29.682914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1319/2612] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:29.690884 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece4cbe-8202-40c8-a10f-433f4d4a46c0 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:29.691189 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece4cbe-8202-40c8-a10f-433f4d4a46c0 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:29.691475 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece4cbe-8202-40c8-a10f-433f4d4a46c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:29.711621 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-7ece4cbe-8202-40c8-a10f-433f4d4a46c0 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:29.712976 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-7ece4cbe-8202-40c8-a10f-433f4d4a46c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:29.721864 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1320/2613] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:21:29.726034 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-459d859a-956e-49a4-9f7a-b8a5a8c50635 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c96f29dc500445ffbd5a2f2af965e16e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:29.726736 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece4cbe-8202-40c8-a10f-433f4d4a46c0 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:29.734167 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-459d859a-956e-49a4-9f7a-b8a5a8c50635 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c96f29dc500445ffbd5a2f2af965e16e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:29.734875 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-459d859a-956e-49a4-9f7a-b8a5a8c50635 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c96f29dc500445ffbd5a2f2af965e16e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:29.737726 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece4cbe-8202-40c8-a10f-433f4d4a46c0 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:29.743401 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05348e9c-bc35-41c6-a3e5-360c537ffc01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:29.744400 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05348e9c-bc35-41c6-a3e5-360c537ffc01 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:29.744559 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-459d859a-956e-49a4-9f7a-b8a5a8c50635 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c96f29dc500445ffbd5a2f2af965e16e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:29.744721 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05348e9c-bc35-41c6-a3e5-360c537ffc01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:29.745056 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05348e9c-bc35-41c6-a3e5-360c537ffc01 admin admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:29.745685 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-459d859a-956e-49a4-9f7a-b8a5a8c50635 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c96f29dc500445ffbd5a2f2af965e16e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:29.745829 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05348e9c-bc35-41c6-a3e5-360c537ffc01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:21:29.746108 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05348e9c-bc35-41c6-a3e5-360c537ffc01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:29.749428 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05348e9c-bc35-41c6-a3e5-360c537ffc01 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:29.760949 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1321/2614] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:21:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:21:29.765571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05348e9c-bc35-41c6-a3e5-360c537ffc01 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:29.767433 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-05348e9c-bc35-41c6-a3e5-360c537ffc01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:29.777751 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1294/2615] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:29 2022] DELETE /identity/v3/users/c96f29dc500445ffbd5a2f2af965e16e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:29.781369 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664ae71-4c3c-4fdf-82d1-1619ec6a6cbd None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:29.781510 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05348e9c-bc35-41c6-a3e5-360c537ffc01 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:29.781704 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664ae71-4c3c-4fdf-82d1-1619ec6a6cbd None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:29.781942 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664ae71-4c3c-4fdf-82d1-1619ec6a6cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:29.810977 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b664ae71-4c3c-4fdf-82d1-1619ec6a6cbd None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:29.812566 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b664ae71-4c3c-4fdf-82d1-1619ec6a6cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:29.842499 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46402a-9e98-4138-b6d0-8f462ec16370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:29.843775 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46402a-9e98-4138-b6d0-8f462ec16370 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:29.844126 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46402a-9e98-4138-b6d0-8f462ec16370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:29.844408 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46402a-9e98-4138-b6d0-8f462ec16370 admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:29.845723 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e46402a-9e98-4138-b6d0-8f462ec16370 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:29.847732 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e46402a-9e98-4138-b6d0-8f462ec16370 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:29.878184 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:29.887825 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1322/2616] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:29 2022] POST /identity/v3/projects => generated 365 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:29.888914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46402a-9e98-4138-b6d0-8f462ec16370 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:29.896392 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1295/2617] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:21:29.906505 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b664ae71-4c3c-4fdf-82d1-1619ec6a6cbd None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:29.908629 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b664ae71-4c3c-4fdf-82d1-1619ec6a6cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:29.987318 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46402a-9e98-4138-b6d0-8f462ec16370 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:29.995516 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fbbef-5a58-4dac-afb5-a8594998d26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:29.996400 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fbbef-5a58-4dac-afb5-a8594998d26e admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:29.996746 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fbbef-5a58-4dac-afb5-a8594998d26e admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:29.997396 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fbbef-5a58-4dac-afb5-a8594998d26e admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:29.998366 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551fbbef-5a58-4dac-afb5-a8594998d26e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:30.000369 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551fbbef-5a58-4dac-afb5-a8594998d26e admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:30.010574 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b664ae71-4c3c-4fdf-82d1-1619ec6a6cbd None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:30.039688 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1323/2618] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:29 2022] POST /identity/v3/users => generated 283 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:30.043845 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fbbef-5a58-4dac-afb5-a8594998d26e admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:30.071398 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6435c8-f24a-4d7a-bdb5-16ad35feca98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:30.071988 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6435c8-f24a-4d7a-bdb5-16ad35feca98 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:30.072251 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6435c8-f24a-4d7a-bdb5-16ad35feca98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:30.072548 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6435c8-f24a-4d7a-bdb5-16ad35feca98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:30.075359 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:30.081819 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6435c8-f24a-4d7a-bdb5-16ad35feca98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:30.083274 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6435c8-f24a-4d7a-bdb5-16ad35feca98 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:30.093050 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1296/2619] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:21:30.093859 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6435c8-f24a-4d7a-bdb5-16ad35feca98 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:30.104639 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6435c8-f24a-4d7a-bdb5-16ad35feca98 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:30.105004 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fbbef-5a58-4dac-afb5-a8594998d26e admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:30.109218 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be57e0-2299-4b2c-95b9-38be3d73b19b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:30.109541 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16440cd-d319-419f-92e5-0911f914f8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:30.110184 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be57e0-2299-4b2c-95b9-38be3d73b19b service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:30.110295 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16440cd-d319-419f-92e5-0911f914f8e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:30.110489 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be57e0-2299-4b2c-95b9-38be3d73b19b service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:30.110919 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be57e0-2299-4b2c-95b9-38be3d73b19b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:30.111039 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16440cd-d319-419f-92e5-0911f914f8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:30.111218 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16440cd-d319-419f-92e5-0911f914f8e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:30.111925 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16440cd-d319-419f-92e5-0911f914f8e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:30.113557 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16440cd-d319-419f-92e5-0911f914f8e1 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:30.113839 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-c1be57e0-2299-4b2c-95b9-38be3d73b19b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:30.125053 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1297/2620] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:30.129465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16440cd-d319-419f-92e5-0911f914f8e1 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:30.130206 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1be57e0-2299-4b2c-95b9-38be3d73b19b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:30.131787 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1be57e0-2299-4b2c-95b9-38be3d73b19b service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:30.141957 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16440cd-d319-419f-92e5-0911f914f8e1 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:30.144161 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1324/2621] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:21:30.147897 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e62ad3-2e96-46a0-81fc-d62f38ba74d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:30.148660 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e62ad3-2e96-46a0-81fc-d62f38ba74d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:30.149126 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e62ad3-2e96-46a0-81fc-d62f38ba74d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:30.149470 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e62ad3-2e96-46a0-81fc-d62f38ba74d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:30.150184 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e62ad3-2e96-46a0-81fc-d62f38ba74d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:30.151953 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e62ad3-2e96-46a0-81fc-d62f38ba74d2 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:30.167161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1298/2622] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:30.168368 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be57e0-2299-4b2c-95b9-38be3d73b19b service neutron] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:30.182240 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be57e0-2299-4b2c-95b9-38be3d73b19b service neutron] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:30.188950 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43b4e1-6dc7-4ce6-934c-d5e9d1b47b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:30.189641 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43b4e1-6dc7-4ce6-934c-d5e9d1b47b75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:30.189927 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43b4e1-6dc7-4ce6-934c-d5e9d1b47b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:30.190711 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43b4e1-6dc7-4ce6-934c-d5e9d1b47b75 admin admin] PATH_INFO: `/v3/projects/0a5c2a0f8e014c3bbbd5a6b2123ad6a3/users/8b4acd252f0d4cdba3441b74a675db98/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:30.209131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a43b4e1-6dc7-4ce6-934c-d5e9d1b47b75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a5c2a0f8e014c3bbbd5a6b2123ad6a3, user_id=8b4acd252f0d4cdba3441b74a675db98, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:30.211126 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a43b4e1-6dc7-4ce6-934c-d5e9d1b47b75 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:30.225986 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1325/2623] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:30 2022] PUT /identity/v3/projects/0a5c2a0f8e014c3bbbd5a6b2123ad6a3/users/8b4acd252f0d4cdba3441b74a675db98/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:30.236378 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef0e8c-a6e3-466d-8a77-e6881111f371 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:30.237035 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef0e8c-a6e3-466d-8a77-e6881111f371 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:30.237320 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef0e8c-a6e3-466d-8a77-e6881111f371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:30.267385 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-0cef0e8c-a6e3-466d-8a77-e6881111f371 None None] MFA Rules not processed for user `8b4acd252f0d4cdba3441b74a675db98`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:30.269246 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0cef0e8c-a6e3-466d-8a77-e6881111f371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:30.358076 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1299/2624] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:30 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:30.359804 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43b4e1-6dc7-4ce6-934c-d5e9d1b47b75 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:30.451451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43b4e1-6dc7-4ce6-934c-d5e9d1b47b75 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:30.467945 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad768f52-12a3-4f02-9ac5-56acaed65f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:30.468833 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad768f52-12a3-4f02-9ac5-56acaed65f0c admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:30.469255 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad768f52-12a3-4f02-9ac5-56acaed65f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:30.469516 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad768f52-12a3-4f02-9ac5-56acaed65f0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:30.470525 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad768f52-12a3-4f02-9ac5-56acaed65f0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:30.473840 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad768f52-12a3-4f02-9ac5-56acaed65f0c admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:30.507874 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1326/2625] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:30 2022] POST /identity/v3/projects => generated 363 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:21:30.508823 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef0e8c-a6e3-466d-8a77-e6881111f371 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:30.520136 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef0e8c-a6e3-466d-8a77-e6881111f371 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:30.545451 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:30.567529 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad768f52-12a3-4f02-9ac5-56acaed65f0c admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:30.569839 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-ad768f52-12a3-4f02-9ac5-56acaed65f0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:30.595197 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9827960b-e68d-4302-a9bc-682cd403340b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:30.595870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9827960b-e68d-4302-a9bc-682cd403340b admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:30.596524 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9827960b-e68d-4302-a9bc-682cd403340b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:30.596524 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9827960b-e68d-4302-a9bc-682cd403340b admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:30.597433 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9827960b-e68d-4302-a9bc-682cd403340b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:30.600251 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9827960b-e68d-4302-a9bc-682cd403340b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:30.627108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1300/2626] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:30 2022] POST /identity/v3/users => generated 288 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:21:30.630877 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9827960b-e68d-4302-a9bc-682cd403340b admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:30.648864 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9827960b-e68d-4302-a9bc-682cd403340b admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:30.655749 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9c91e-84e0-40b9-9a58-0507df2829c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:30.656560 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9c91e-84e0-40b9-9a58-0507df2829c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:30.656945 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9c91e-84e0-40b9-9a58-0507df2829c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:30.657323 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9c91e-84e0-40b9-9a58-0507df2829c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:30.658124 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f9c91e-84e0-40b9-9a58-0507df2829c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:30.660266 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f9c91e-84e0-40b9-9a58-0507df2829c6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:30.669666 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad768f52-12a3-4f02-9ac5-56acaed65f0c admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:30.672883 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1301/2627] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:30 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:21:30.677375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9c91e-84e0-40b9-9a58-0507df2829c6 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:30.678412 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea13702-2ea5-4683-9f5e-bf744a6779f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:30.679061 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea13702-2ea5-4683-9f5e-bf744a6779f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:30.679360 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea13702-2ea5-4683-9f5e-bf744a6779f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:30.679629 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea13702-2ea5-4683-9f5e-bf744a6779f7 admin admin] PATH_INFO: `/v3/projects/4de236911e6c4b7e820700b474b0814c` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:30.681105 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea13702-2ea5-4683-9f5e-bf744a6779f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4de236911e6c4b7e820700b474b0814c)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:30.683380 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea13702-2ea5-4683-9f5e-bf744a6779f7 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:30.691440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9c91e-84e0-40b9-9a58-0507df2829c6 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:30.700129 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a4092e-0921-4e00-9820-6a490688cbdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:30.700867 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4092e-0921-4e00-9820-6a490688cbdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:30.701156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4092e-0921-4e00-9820-6a490688cbdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:30.702108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4092e-0921-4e00-9820-6a490688cbdc admin admin] PATH_INFO: `/v3/projects/efefa095b5a14057aaadc08cc7fc9787/users/e55df4d286ba4395831cd9492aff1f6a/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:30.719862 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a4092e-0921-4e00-9820-6a490688cbdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=efefa095b5a14057aaadc08cc7fc9787, user_id=e55df4d286ba4395831cd9492aff1f6a, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:30.722528 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a4092e-0921-4e00-9820-6a490688cbdc admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:30.734738 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:30.737359 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1302/2628] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:30 2022] PUT /identity/v3/projects/efefa095b5a14057aaadc08cc7fc9787/users/e55df4d286ba4395831cd9492aff1f6a/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:30.742583 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a4092e-0921-4e00-9820-6a490688cbdc admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:30.757623 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-8ea13702-2ea5-4683-9f5e-bf744a6779f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4de236911e6c4b7e820700b474b0814c', 'request_id': 'req-8ea13702-2ea5-4683-9f5e-bf744a6779f7'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:30.765111 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-8ea13702-2ea5-4683-9f5e-bf744a6779f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4de236911e6c4b7e820700b474b0814c', 'request_id': 'req-8ea13702-2ea5-4683-9f5e-bf744a6779f7'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:30.774073 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-8ea13702-2ea5-4683-9f5e-bf744a6779f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4de236911e6c4b7e820700b474b0814c', 'request_id': 'req-8ea13702-2ea5-4683-9f5e-bf744a6779f7'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:30.793045 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-8ea13702-2ea5-4683-9f5e-bf744a6779f7 admin admin] The token cache is being invalidate because project 4de236911e6c4b7e820700b474b0814c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:30.793345 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-8ea13702-2ea5-4683-9f5e-bf744a6779f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:30.794042 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-8ea13702-2ea5-4683-9f5e-bf744a6779f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:30.795902 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1327/2629] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:30 2022] DELETE /identity/v3/projects/4de236911e6c4b7e820700b474b0814c => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:30.801673 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea13702-2ea5-4683-9f5e-bf744a6779f7 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:30.803852 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-8ea13702-2ea5-4683-9f5e-bf744a6779f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:30.830802 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a4092e-0921-4e00-9820-6a490688cbdc admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:30.889668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea13702-2ea5-4683-9f5e-bf744a6779f7 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:30.894079 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1390989-5b51-4cb0-b0a9-fb98e7a26776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:30.894456 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb42899-9861-471a-83f1-83dbd401c3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:30.894869 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1390989-5b51-4cb0-b0a9-fb98e7a26776 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:30.895107 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb42899-9861-471a-83f1-83dbd401c3c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:30.895233 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1390989-5b51-4cb0-b0a9-fb98e7a26776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:30.895392 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb42899-9861-471a-83f1-83dbd401c3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:30.895624 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1390989-5b51-4cb0-b0a9-fb98e7a26776 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:30.895765 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb42899-9861-471a-83f1-83dbd401c3c5 admin admin] PATH_INFO: `/v3/users/cee2d65f84924998b642954c037121df` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:30.896431 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1390989-5b51-4cb0-b0a9-fb98e7a26776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:30.897068 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb42899-9861-471a-83f1-83dbd401c3c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cee2d65f84924998b642954c037121df)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:30.898249 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1390989-5b51-4cb0-b0a9-fb98e7a26776 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:30.898980 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb42899-9861-471a-83f1-83dbd401c3c5 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:30.912381 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1303/2630] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:30 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:30.913462 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1390989-5b51-4cb0-b0a9-fb98e7a26776 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:30.916162 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c1390989-5b51-4cb0-b0a9-fb98e7a26776 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:30.930639 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1390989-5b51-4cb0-b0a9-fb98e7a26776 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:30.936988 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda1d83-92ce-411e-a3fa-d0505d877cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:30.937990 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda1d83-92ce-411e-a3fa-d0505d877cfb admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:30.938406 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda1d83-92ce-411e-a3fa-d0505d877cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:30.938769 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda1d83-92ce-411e-a3fa-d0505d877cfb admin admin] PATH_INFO: `/v3/projects/efefa095b5a14057aaadc08cc7fc9787/users/e55df4d286ba4395831cd9492aff1f6a/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:30.941221 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebda1d83-92ce-411e-a3fa-d0505d877cfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=efefa095b5a14057aaadc08cc7fc9787, user_id=e55df4d286ba4395831cd9492aff1f6a, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:30.944140 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebda1d83-92ce-411e-a3fa-d0505d877cfb admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:30.955698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-8eb42899-9861-471a-83f1-83dbd401c3c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cee2d65f84924998b642954c037121df'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:30.956443 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-8eb42899-9861-471a-83f1-83dbd401c3c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cee2d65f84924998b642954c037121df'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:30.966550 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1304/2631] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:30 2022] PUT /identity/v3/projects/efefa095b5a14057aaadc08cc7fc9787/users/e55df4d286ba4395831cd9492aff1f6a/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:30.968454 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf67d4-bac3-47a9-ab5d-261cb53eab1d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:30.968774 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf67d4-bac3-47a9-ab5d-261cb53eab1d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:30.969234 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf67d4-bac3-47a9-ab5d-261cb53eab1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:30.981699 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-8eb42899-9861-471a-83f1-83dbd401c3c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cee2d65f84924998b642954c037121df'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:30.985756 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:30.991501 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-8eb42899-9861-471a-83f1-83dbd401c3c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cee2d65f84924998b642954c037121df'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:30.992656 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-8eb42899-9861-471a-83f1-83dbd401c3c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cee2d65f84924998b642954c037121df'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:31.004424 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-62cf67d4-bac3-47a9-ab5d-261cb53eab1d None None] MFA Rules not processed for user `e55df4d286ba4395831cd9492aff1f6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:31.006346 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-62cf67d4-bac3-47a9-ab5d-261cb53eab1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:31.008609 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1328/2632] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:30 2022] DELETE /identity/v3/users/cee2d65f84924998b642954c037121df => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:31.103015 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb42899-9861-471a-83f1-83dbd401c3c5 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:31.103791 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1305/2633] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:30 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:31.104905 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-8eb42899-9861-471a-83f1-83dbd401c3c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:31.169502 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:31.189351 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb42899-9861-471a-83f1-83dbd401c3c5 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:31.194530 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c1e422-d04f-4b75-94a1-cc92d3443676 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:31.195174 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1e422-d04f-4b75-94a1-cc92d3443676 service cinder] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:31.195458 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1e422-d04f-4b75-94a1-cc92d3443676 service cinder] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:31.195733 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1e422-d04f-4b75-94a1-cc92d3443676 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:31.206111 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c1e422-d04f-4b75-94a1-cc92d3443676 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:31.207549 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c1e422-d04f-4b75-94a1-cc92d3443676 service cinder] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:31.218905 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1329/2634] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:31 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:31.322879 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cf67d4-bac3-47a9-ab5d-261cb53eab1d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:31.328708 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c1e422-d04f-4b75-94a1-cc92d3443676 service cinder] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:31.331726 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-64c1e422-d04f-4b75-94a1-cc92d3443676 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:31.349541 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cf67d4-bac3-47a9-ab5d-261cb53eab1d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:31.359163 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f021b7-0e8e-435f-96e1-5c65bc7ba322 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:31.360033 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f021b7-0e8e-435f-96e1-5c65bc7ba322 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:31.360523 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f021b7-0e8e-435f-96e1-5c65bc7ba322 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:31.364313 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f021b7-0e8e-435f-96e1-5c65bc7ba322 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:31.369338 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a2f021b7-0e8e-435f-96e1-5c65bc7ba322 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:31.439660 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c1e422-d04f-4b75-94a1-cc92d3443676 service cinder] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:31.448228 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2da2e-f4ac-4aa6-a750-a1fa698bb568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:31.449402 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2da2e-f4ac-4aa6-a750-a1fa698bb568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:31.449402 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2da2e-f4ac-4aa6-a750-a1fa698bb568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:31.449680 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2da2e-f4ac-4aa6-a750-a1fa698bb568 admin admin] PATH_INFO: `/v3/projects/828263d2c2134eba9781c88af470b197` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:31.450975 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db2da2e-f4ac-4aa6-a750-a1fa698bb568 admin admin] RBAC: Authorizing `identity:delete_project(project_id=828263d2c2134eba9781c88af470b197)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:31.451887 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f021b7-0e8e-435f-96e1-5c65bc7ba322 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:31.453380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f021b7-0e8e-435f-96e1-5c65bc7ba322 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:31.453633 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db2da2e-f4ac-4aa6-a750-a1fa698bb568 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:31.467907 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1306/2635] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:31 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:31.524369 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-1db2da2e-f4ac-4aa6-a750-a1fa698bb568 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '828263d2c2134eba9781c88af470b197', 'request_id': 'req-1db2da2e-f4ac-4aa6-a750-a1fa698bb568'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:31.536387 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-1db2da2e-f4ac-4aa6-a750-a1fa698bb568 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '828263d2c2134eba9781c88af470b197', 'request_id': 'req-1db2da2e-f4ac-4aa6-a750-a1fa698bb568'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:31.547693 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-1db2da2e-f4ac-4aa6-a750-a1fa698bb568 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '828263d2c2134eba9781c88af470b197', 'request_id': 'req-1db2da2e-f4ac-4aa6-a750-a1fa698bb568'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:31.565703 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-1db2da2e-f4ac-4aa6-a750-a1fa698bb568 admin admin] The token cache is being invalidate because project 828263d2c2134eba9781c88af470b197 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:31.566295 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-1db2da2e-f4ac-4aa6-a750-a1fa698bb568 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:31.567228 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-1db2da2e-f4ac-4aa6-a750-a1fa698bb568 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:31.570503 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1330/2636] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:31 2022] DELETE /identity/v3/projects/828263d2c2134eba9781c88af470b197 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:31.575705 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a68a20-99a9-4fb5-b1aa-921b4b28b817 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:31.576278 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a68a20-99a9-4fb5-b1aa-921b4b28b817 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:31.576661 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a68a20-99a9-4fb5-b1aa-921b4b28b817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:31.600405 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-63a68a20-99a9-4fb5-b1aa-921b4b28b817 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:31.601806 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-63a68a20-99a9-4fb5-b1aa-921b4b28b817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:31.676072 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1307/2637] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:31.687511 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7410d-1980-488b-8b6d-edf949585351 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:31.687955 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7410d-1980-488b-8b6d-edf949585351 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:31.688510 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7410d-1980-488b-8b6d-edf949585351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:31.714460 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-b3c7410d-1980-488b-8b6d-edf949585351 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:31.715758 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-b3c7410d-1980-488b-8b6d-edf949585351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:31.725797 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1331/2638] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:31.730031 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a68a20-99a9-4fb5-b1aa-921b4b28b817 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:31.742220 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a68a20-99a9-4fb5-b1aa-921b4b28b817 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:31.747199 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82878554-3fe9-44cd-bded-36288fc99106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:31.747835 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82878554-3fe9-44cd-bded-36288fc99106 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:31.748116 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82878554-3fe9-44cd-bded-36288fc99106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:31.748458 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82878554-3fe9-44cd-bded-36288fc99106 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:31.749199 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82878554-3fe9-44cd-bded-36288fc99106 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:21:31.749514 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82878554-3fe9-44cd-bded-36288fc99106 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:31.751260 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82878554-3fe9-44cd-bded-36288fc99106 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:31.766778 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1308/2639] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:21:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:21:31.771566 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7410d-1980-488b-8b6d-edf949585351 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:31.782217 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7410d-1980-488b-8b6d-edf949585351 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:31.789001 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798796c8-45e4-41b0-8ae2-167ab5ad7314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:31.789871 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798796c8-45e4-41b0-8ae2-167ab5ad7314 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:31.790584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798796c8-45e4-41b0-8ae2-167ab5ad7314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:31.791080 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798796c8-45e4-41b0-8ae2-167ab5ad7314 admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:31.792407 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798796c8-45e4-41b0-8ae2-167ab5ad7314 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:31.794599 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798796c8-45e4-41b0-8ae2-167ab5ad7314 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:31.837574 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1332/2640] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:31 2022] POST /identity/v3/projects => generated 379 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:31.841255 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82878554-3fe9-44cd-bded-36288fc99106 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:31.853238 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82878554-3fe9-44cd-bded-36288fc99106 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:31.871217 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:31.940202 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5dcc07-49d0-4a2e-9607-593f4be902eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:31.941135 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5dcc07-49d0-4a2e-9607-593f4be902eb admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:31.941511 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5dcc07-49d0-4a2e-9607-593f4be902eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:31.941803 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5dcc07-49d0-4a2e-9607-593f4be902eb admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:31.943139 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5dcc07-49d0-4a2e-9607-593f4be902eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:31.945081 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5dcc07-49d0-4a2e-9607-593f4be902eb admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:31.971485 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1309/2641] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:31 2022] POST /identity/v3/users => generated 290 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:31.975444 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798796c8-45e4-41b0-8ae2-167ab5ad7314 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:31.987317 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798796c8-45e4-41b0-8ae2-167ab5ad7314 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:31.995034 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f92ca77-7459-48cc-81bf-34617cb3e817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:31.995784 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f92ca77-7459-48cc-81bf-34617cb3e817 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:31.996148 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f92ca77-7459-48cc-81bf-34617cb3e817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:31.996537 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f92ca77-7459-48cc-81bf-34617cb3e817 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:31.997366 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f92ca77-7459-48cc-81bf-34617cb3e817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:31.999210 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f92ca77-7459-48cc-81bf-34617cb3e817 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:32.012518 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1333/2642] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:32.015184 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5dcc07-49d0-4a2e-9607-593f4be902eb admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:32.027043 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5dcc07-49d0-4a2e-9607-593f4be902eb admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:32.037164 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c8bd7-520b-4e82-a632-8e30a52dab0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:32.037164 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c8bd7-520b-4e82-a632-8e30a52dab0c admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:32.037164 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c8bd7-520b-4e82-a632-8e30a52dab0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:32.037844 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c8bd7-520b-4e82-a632-8e30a52dab0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:32.038577 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927c8bd7-520b-4e82-a632-8e30a52dab0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:32.040441 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927c8bd7-520b-4e82-a632-8e30a52dab0c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:32.055790 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1310/2643] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:32.060273 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f92ca77-7459-48cc-81bf-34617cb3e817 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:32.071697 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f92ca77-7459-48cc-81bf-34617cb3e817 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:32.076534 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab36780f-6f4a-4001-bb6d-b336b276fa7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:32.077307 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36780f-6f4a-4001-bb6d-b336b276fa7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:32.077664 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36780f-6f4a-4001-bb6d-b336b276fa7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:32.078019 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36780f-6f4a-4001-bb6d-b336b276fa7c admin admin] PATH_INFO: `/v3/projects/516c1018e65840c784b3bb4692e2e42d/users/304b382d2fa44ee3ae7d68669a779108/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:32.094005 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab36780f-6f4a-4001-bb6d-b336b276fa7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=516c1018e65840c784b3bb4692e2e42d, user_id=304b382d2fa44ee3ae7d68669a779108, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:32.096827 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab36780f-6f4a-4001-bb6d-b336b276fa7c admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:32.121486 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1334/2644] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:32 2022] PUT /identity/v3/projects/516c1018e65840c784b3bb4692e2e42d/users/304b382d2fa44ee3ae7d68669a779108/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:32.124909 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d510715-58af-458e-a8c8-17e5c8efc518 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:32.125206 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d510715-58af-458e-a8c8-17e5c8efc518 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:32.125487 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d510715-58af-458e-a8c8-17e5c8efc518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:32.157344 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-9d510715-58af-458e-a8c8-17e5c8efc518 None None] MFA Rules not processed for user `304b382d2fa44ee3ae7d68669a779108`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:32.158691 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9d510715-58af-458e-a8c8-17e5c8efc518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:32.244157 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1311/2645] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:32.246135 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab36780f-6f4a-4001-bb6d-b336b276fa7c admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:32.260468 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:32.336100 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab36780f-6f4a-4001-bb6d-b336b276fa7c admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:32.341329 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc64f92-95a3-4d18-a035-b47ab819e870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:32.341977 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc64f92-95a3-4d18-a035-b47ab819e870 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:32.342292 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc64f92-95a3-4d18-a035-b47ab819e870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:32.342571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc64f92-95a3-4d18-a035-b47ab819e870 admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:32.343252 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc64f92-95a3-4d18-a035-b47ab819e870 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:32.345217 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc64f92-95a3-4d18-a035-b47ab819e870 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:32.376609 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1335/2646] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:32 2022] POST /identity/v3/projects => generated 379 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:21:32.380597 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d510715-58af-458e-a8c8-17e5c8efc518 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:32.391699 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d510715-58af-458e-a8c8-17e5c8efc518 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:32.457880 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5dedff-f931-44b2-834e-8f23a678a043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:32.458569 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5dedff-f931-44b2-834e-8f23a678a043 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:32.458855 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5dedff-f931-44b2-834e-8f23a678a043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:32.459133 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5dedff-f931-44b2-834e-8f23a678a043 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:32.459804 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5dedff-f931-44b2-834e-8f23a678a043 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:32.461749 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5dedff-f931-44b2-834e-8f23a678a043 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:32.488529 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1312/2647] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:32 2022] POST /identity/v3/users => generated 296 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:32.492935 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc64f92-95a3-4d18-a035-b47ab819e870 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:32.505877 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc64f92-95a3-4d18-a035-b47ab819e870 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:32.510863 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a930d-3e98-40a5-aa00-509959898d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:32.511624 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a930d-3e98-40a5-aa00-509959898d2f admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:32.511978 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a930d-3e98-40a5-aa00-509959898d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:32.512393 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a930d-3e98-40a5-aa00-509959898d2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:32.513254 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4a930d-3e98-40a5-aa00-509959898d2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:32.514970 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4a930d-3e98-40a5-aa00-509959898d2f admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:32.535984 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1336/2648] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:32.541026 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5dedff-f931-44b2-834e-8f23a678a043 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:32.565846 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5dedff-f931-44b2-834e-8f23a678a043 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:32.570764 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa04c1e-8fbd-43f8-8334-74dfed83acc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:32.571731 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa04c1e-8fbd-43f8-8334-74dfed83acc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:32.572566 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa04c1e-8fbd-43f8-8334-74dfed83acc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:32.572566 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa04c1e-8fbd-43f8-8334-74dfed83acc4 admin admin] PATH_INFO: `/v3/projects/1392802df1f349bcb788d145378343ba/users/74c71312fc844def840d607f02680222/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:32.585624 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:32.590322 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa04c1e-8fbd-43f8-8334-74dfed83acc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1392802df1f349bcb788d145378343ba, user_id=74c71312fc844def840d607f02680222, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:32.594249 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa04c1e-8fbd-43f8-8334-74dfed83acc4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:32.609533 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1313/2649] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:32 2022] PUT /identity/v3/projects/1392802df1f349bcb788d145378343ba/users/74c71312fc844def840d607f02680222/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:32.613329 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a930d-3e98-40a5-aa00-509959898d2f admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:32.624386 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a930d-3e98-40a5-aa00-509959898d2f admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:32.692716 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff8c8d-ec60-4911-8b4b-d515f9f3b5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:32.693391 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff8c8d-ec60-4911-8b4b-d515f9f3b5ff admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:32.693677 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff8c8d-ec60-4911-8b4b-d515f9f3b5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:32.693961 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff8c8d-ec60-4911-8b4b-d515f9f3b5ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:32.695063 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ff8c8d-ec60-4911-8b4b-d515f9f3b5ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:32.696992 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ff8c8d-ec60-4911-8b4b-d515f9f3b5ff admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:32.708313 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1337/2650] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:32.712231 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa04c1e-8fbd-43f8-8334-74dfed83acc4 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:32.722832 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa04c1e-8fbd-43f8-8334-74dfed83acc4 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:32.728007 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22516a1-d161-4e2e-a119-f4e6ad0fe806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:32.728719 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22516a1-d161-4e2e-a119-f4e6ad0fe806 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:32.729011 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22516a1-d161-4e2e-a119-f4e6ad0fe806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:32.729982 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22516a1-d161-4e2e-a119-f4e6ad0fe806 admin admin] PATH_INFO: `/v3/projects/1392802df1f349bcb788d145378343ba/users/74c71312fc844def840d607f02680222/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:32.731405 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22516a1-d161-4e2e-a119-f4e6ad0fe806 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1392802df1f349bcb788d145378343ba, user_id=74c71312fc844def840d607f02680222, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:32.734396 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22516a1-d161-4e2e-a119-f4e6ad0fe806 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:32.755225 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1314/2651] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:32 2022] PUT /identity/v3/projects/1392802df1f349bcb788d145378343ba/users/74c71312fc844def840d607f02680222/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:32.765582 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc5125-9dc8-40ac-b5fa-a7c61475d8b0 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:32.765958 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc5125-9dc8-40ac-b5fa-a7c61475d8b0 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:32.766315 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc5125-9dc8-40ac-b5fa-a7c61475d8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:32.806901 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-5edc5125-9dc8-40ac-b5fa-a7c61475d8b0 None None] MFA Rules not processed for user `74c71312fc844def840d607f02680222`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:32.808246 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-5edc5125-9dc8-40ac-b5fa-a7c61475d8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:32.925441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1338/2652] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:32 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:21:32.952844 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22516a1-d161-4e2e-a119-f4e6ad0fe806 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:32.955870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c22516a1-d161-4e2e-a119-f4e6ad0fe806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:33.032152 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22516a1-d161-4e2e-a119-f4e6ad0fe806 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:33.036184 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16717af7-7be7-463c-b761-dcf9800339be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:33.036849 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16717af7-7be7-463c-b761-dcf9800339be service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:33.037131 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16717af7-7be7-463c-b761-dcf9800339be service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:33.037405 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16717af7-7be7-463c-b761-dcf9800339be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:33.049557 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16717af7-7be7-463c-b761-dcf9800339be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:33.051267 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16717af7-7be7-463c-b761-dcf9800339be service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:33.069264 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1315/2653] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:32 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:33.376518 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc5125-9dc8-40ac-b5fa-a7c61475d8b0 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:33.378513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-5edc5125-9dc8-40ac-b5fa-a7c61475d8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:33.388672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:33.393064 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc5125-9dc8-40ac-b5fa-a7c61475d8b0 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:33.403947 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af9c48-4782-4e11-bf7c-99181ab67418 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:33.404805 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af9c48-4782-4e11-bf7c-99181ab67418 service cinder] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:33.405099 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af9c48-4782-4e11-bf7c-99181ab67418 service cinder] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:33.405432 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af9c48-4782-4e11-bf7c-99181ab67418 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:33.410160 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-d3af9c48-4782-4e11-bf7c-99181ab67418 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:33.541038 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af9c48-4782-4e11-bf7c-99181ab67418 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:33.543592 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af9c48-4782-4e11-bf7c-99181ab67418 service cinder] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:33.563126 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1339/2654] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:33.696425 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16717af7-7be7-463c-b761-dcf9800339be service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:33.846204 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16717af7-7be7-463c-b761-dcf9800339be service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:33.856983 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c74002-7ad5-448c-9410-88d46e85f065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:33.858978 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c74002-7ad5-448c-9410-88d46e85f065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:33.859445 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c74002-7ad5-448c-9410-88d46e85f065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:33.860473 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c74002-7ad5-448c-9410-88d46e85f065 admin admin] PATH_INFO: `/v3/users/304b382d2fa44ee3ae7d68669a779108` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:33.862560 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c74002-7ad5-448c-9410-88d46e85f065 admin admin] RBAC: Authorizing `identity:delete_user(user_id=304b382d2fa44ee3ae7d68669a779108)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:33.868004 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c74002-7ad5-448c-9410-88d46e85f065 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:33.947102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f6c74002-7ad5-448c-9410-88d46e85f065 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '304b382d2fa44ee3ae7d68669a779108'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:33.957312 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f6c74002-7ad5-448c-9410-88d46e85f065 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '304b382d2fa44ee3ae7d68669a779108'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:33.958942 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f6c74002-7ad5-448c-9410-88d46e85f065 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '304b382d2fa44ee3ae7d68669a779108'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:33.972049 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f6c74002-7ad5-448c-9410-88d46e85f065 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '304b382d2fa44ee3ae7d68669a779108'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:33.973075 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f6c74002-7ad5-448c-9410-88d46e85f065 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '304b382d2fa44ee3ae7d68669a779108'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:34.003100 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:34.012643 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1316/2655] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:33 2022] DELETE /identity/v3/users/304b382d2fa44ee3ae7d68669a779108 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:34.015979 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13252aa-dd55-4e45-b887-2edf16a21ea9 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:34.016884 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13252aa-dd55-4e45-b887-2edf16a21ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:34.016884 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13252aa-dd55-4e45-b887-2edf16a21ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:34.051022 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-f13252aa-dd55-4e45-b887-2edf16a21ea9 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:34.053059 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-f13252aa-dd55-4e45-b887-2edf16a21ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:34.121668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1340/2656] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:34.132500 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c74002-7ad5-448c-9410-88d46e85f065 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:34.133932 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f6c74002-7ad5-448c-9410-88d46e85f065 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:34.172407 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13252aa-dd55-4e45-b887-2edf16a21ea9 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:34.173706 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-f13252aa-dd55-4e45-b887-2edf16a21ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:34.216574 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c74002-7ad5-448c-9410-88d46e85f065 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:34.222002 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfacf06-9166-45e8-aa72-31ed160dec30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:34.222647 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfacf06-9166-45e8-aa72-31ed160dec30 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:34.222945 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfacf06-9166-45e8-aa72-31ed160dec30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:34.223212 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfacf06-9166-45e8-aa72-31ed160dec30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:34.241724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfacf06-9166-45e8-aa72-31ed160dec30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:34.244383 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfacf06-9166-45e8-aa72-31ed160dec30 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:34.271461 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1317/2657] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:34.314050 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13252aa-dd55-4e45-b887-2edf16a21ea9 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:34.328177 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278f073c-db45-4665-81c2-b6b412c18ea1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e294fe796a1a4ac89041a532856ad627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:34.328853 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f073c-db45-4665-81c2-b6b412c18ea1 service placement] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:34.329139 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f073c-db45-4665-81c2-b6b412c18ea1 service placement] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:34.329416 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f073c-db45-4665-81c2-b6b412c18ea1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:34.337654 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-278f073c-db45-4665-81c2-b6b412c18ea1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:34.369021 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278f073c-db45-4665-81c2-b6b412c18ea1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:34.369021 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278f073c-db45-4665-81c2-b6b412c18ea1 service placement] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:34.380772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1341/2658] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:34.780159 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfacf06-9166-45e8-aa72-31ed160dec30 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:34.781803 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-bcfacf06-9166-45e8-aa72-31ed160dec30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:34.801536 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfacf06-9166-45e8-aa72-31ed160dec30 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:34.809716 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a774d40f-0717-45d3-9ea3-1baa7e1d6f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:34.811309 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774d40f-0717-45d3-9ea3-1baa7e1d6f39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:34.825364 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774d40f-0717-45d3-9ea3-1baa7e1d6f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:34.828299 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774d40f-0717-45d3-9ea3-1baa7e1d6f39 admin admin] PATH_INFO: `/v3/projects/516c1018e65840c784b3bb4692e2e42d` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:34.830482 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a774d40f-0717-45d3-9ea3-1baa7e1d6f39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=516c1018e65840c784b3bb4692e2e42d)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:34.833443 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a774d40f-0717-45d3-9ea3-1baa7e1d6f39 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:34.907550 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a774d40f-0717-45d3-9ea3-1baa7e1d6f39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '516c1018e65840c784b3bb4692e2e42d', 'request_id': 'req-a774d40f-0717-45d3-9ea3-1baa7e1d6f39'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:34.918479 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a774d40f-0717-45d3-9ea3-1baa7e1d6f39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '516c1018e65840c784b3bb4692e2e42d', 'request_id': 'req-a774d40f-0717-45d3-9ea3-1baa7e1d6f39'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:34.936823 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a774d40f-0717-45d3-9ea3-1baa7e1d6f39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '516c1018e65840c784b3bb4692e2e42d', 'request_id': 'req-a774d40f-0717-45d3-9ea3-1baa7e1d6f39'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:34.953956 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a774d40f-0717-45d3-9ea3-1baa7e1d6f39 admin admin] The token cache is being invalidate because project 516c1018e65840c784b3bb4692e2e42d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:34.954254 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a774d40f-0717-45d3-9ea3-1baa7e1d6f39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:34.956012 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a774d40f-0717-45d3-9ea3-1baa7e1d6f39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:34.956740 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1318/2659] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:34 2022] DELETE /identity/v3/projects/516c1018e65840c784b3bb4692e2e42d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:34.960777 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278f073c-db45-4665-81c2-b6b412c18ea1 service placement] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:34.971888 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278f073c-db45-4665-81c2-b6b412c18ea1 service placement] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:35.008315 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:35.044760 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49814e96-d15b-4f90-b747-904422cfddc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:35.046253 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49814e96-d15b-4f90-b747-904422cfddc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:35.046791 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49814e96-d15b-4f90-b747-904422cfddc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:35.047901 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49814e96-d15b-4f90-b747-904422cfddc2 admin admin] PATH_INFO: `/v3/users/74c71312fc844def840d607f02680222` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:35.049798 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49814e96-d15b-4f90-b747-904422cfddc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74c71312fc844def840d607f02680222)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:35.052760 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49814e96-d15b-4f90-b747-904422cfddc2 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:35.107287 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-49814e96-d15b-4f90-b747-904422cfddc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74c71312fc844def840d607f02680222'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:35.107994 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-49814e96-d15b-4f90-b747-904422cfddc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74c71312fc844def840d607f02680222'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:35.120596 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-49814e96-d15b-4f90-b747-904422cfddc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74c71312fc844def840d607f02680222'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:35.130101 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-49814e96-d15b-4f90-b747-904422cfddc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74c71312fc844def840d607f02680222'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:35.131052 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-49814e96-d15b-4f90-b747-904422cfddc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74c71312fc844def840d607f02680222'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:35.145334 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1342/2660] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:34 2022] DELETE /identity/v3/users/74c71312fc844def840d607f02680222 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:35.566279 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a774d40f-0717-45d3-9ea3-1baa7e1d6f39 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:35.568370 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a774d40f-0717-45d3-9ea3-1baa7e1d6f39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:35.587309 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a774d40f-0717-45d3-9ea3-1baa7e1d6f39 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:35.670352 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30004e4-094a-4fd4-a5a4-403b2b57af0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:35.670832 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30004e4-094a-4fd4-a5a4-403b2b57af0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:35.671117 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30004e4-094a-4fd4-a5a4-403b2b57af0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:35.671409 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30004e4-094a-4fd4-a5a4-403b2b57af0b admin admin] PATH_INFO: `/v3/projects/1392802df1f349bcb788d145378343ba` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:35.672979 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30004e4-094a-4fd4-a5a4-403b2b57af0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1392802df1f349bcb788d145378343ba)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:35.675122 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30004e4-094a-4fd4-a5a4-403b2b57af0b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:35.720015 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:35.741464 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f30004e4-094a-4fd4-a5a4-403b2b57af0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1392802df1f349bcb788d145378343ba', 'request_id': 'req-f30004e4-094a-4fd4-a5a4-403b2b57af0b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:35.750724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f30004e4-094a-4fd4-a5a4-403b2b57af0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1392802df1f349bcb788d145378343ba', 'request_id': 'req-f30004e4-094a-4fd4-a5a4-403b2b57af0b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:35.760280 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f30004e4-094a-4fd4-a5a4-403b2b57af0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1392802df1f349bcb788d145378343ba', 'request_id': 'req-f30004e4-094a-4fd4-a5a4-403b2b57af0b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:35.771920 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f30004e4-094a-4fd4-a5a4-403b2b57af0b admin admin] The token cache is being invalidate because project 1392802df1f349bcb788d145378343ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:35.772222 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f30004e4-094a-4fd4-a5a4-403b2b57af0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:35.772920 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f30004e4-094a-4fd4-a5a4-403b2b57af0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:35.774597 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1319/2661] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:35 2022] DELETE /identity/v3/projects/1392802df1f349bcb788d145378343ba => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:35.779667 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3931f-1dea-467c-9844-c0983974cf82 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:35.780042 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3931f-1dea-467c-9844-c0983974cf82 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:35.780405 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3931f-1dea-467c-9844-c0983974cf82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:35.800391 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-5ea3931f-1dea-467c-9844-c0983974cf82 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:35.802325 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-5ea3931f-1dea-467c-9844-c0983974cf82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:35.887745 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1343/2662] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:21:35.900351 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea85cb9-fc0c-4214-916a-86ef8b9e85ee None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:35.900351 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea85cb9-fc0c-4214-916a-86ef8b9e85ee None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:35.900351 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea85cb9-fc0c-4214-916a-86ef8b9e85ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:35.919435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-7ea85cb9-fc0c-4214-916a-86ef8b9e85ee None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:35.920907 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7ea85cb9-fc0c-4214-916a-86ef8b9e85ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:35.930611 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1320/2663] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:21:35.935759 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3931f-1dea-467c-9844-c0983974cf82 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:35.946664 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3931f-1dea-467c-9844-c0983974cf82 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:35.953191 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ccb68-dc1b-4651-a7ba-8951b3361039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:35.953191 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ccb68-dc1b-4651-a7ba-8951b3361039 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:35.953474 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ccb68-dc1b-4651-a7ba-8951b3361039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:35.953589 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ccb68-dc1b-4651-a7ba-8951b3361039 admin admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:35.954289 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0ccb68-dc1b-4651-a7ba-8951b3361039 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:21:35.954597 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0ccb68-dc1b-4651-a7ba-8951b3361039 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:35.956529 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0ccb68-dc1b-4651-a7ba-8951b3361039 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:35.972721 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1344/2664] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:21:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:21:35.973569 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea85cb9-fc0c-4214-916a-86ef8b9e85ee None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:35.986156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea85cb9-fc0c-4214-916a-86ef8b9e85ee None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:35.991937 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42219485-e75a-4557-8d0b-0e7ea626062b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:35.992208 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42219485-e75a-4557-8d0b-0e7ea626062b admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:35.992526 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42219485-e75a-4557-8d0b-0e7ea626062b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:35.992818 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42219485-e75a-4557-8d0b-0e7ea626062b admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:35.993522 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42219485-e75a-4557-8d0b-0e7ea626062b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:35.996051 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42219485-e75a-4557-8d0b-0e7ea626062b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:36.034369 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1321/2665] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:35 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:36.038682 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ccb68-dc1b-4651-a7ba-8951b3361039 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:36.050237 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ccb68-dc1b-4651-a7ba-8951b3361039 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:36.101874 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:36.130330 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a2904-ed94-4d66-a900-71b5b4302568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:36.131459 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a2904-ed94-4d66-a900-71b5b4302568 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:36.132065 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a2904-ed94-4d66-a900-71b5b4302568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:36.132520 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a2904-ed94-4d66-a900-71b5b4302568 admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:36.134044 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a2904-ed94-4d66-a900-71b5b4302568 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:36.137165 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a2904-ed94-4d66-a900-71b5b4302568 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:36.152172 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42219485-e75a-4557-8d0b-0e7ea626062b admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:36.155004 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-42219485-e75a-4557-8d0b-0e7ea626062b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:36.177737 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1345/2666] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:36 2022] POST /identity/v3/users => generated 292 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:36.182110 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a2904-ed94-4d66-a900-71b5b4302568 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:36.184618 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42219485-e75a-4557-8d0b-0e7ea626062b admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:36.194056 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a2904-ed94-4d66-a900-71b5b4302568 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:36.198120 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3009067-d85a-4172-a66b-818df6b8b5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:36.198898 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3009067-d85a-4172-a66b-818df6b8b5d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:36.199241 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3009067-d85a-4172-a66b-818df6b8b5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:36.199505 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205f034-86be-4830-834b-a0315db7dad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:36.199655 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3009067-d85a-4172-a66b-818df6b8b5d4 admin admin] PATH_INFO: `/v3/users/45782c948da04daab84db1d8de609daf` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:36.200396 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205f034-86be-4830-834b-a0315db7dad2 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:36.200775 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205f034-86be-4830-834b-a0315db7dad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:36.201082 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3009067-d85a-4172-a66b-818df6b8b5d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45782c948da04daab84db1d8de609daf)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:36.201218 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205f034-86be-4830-834b-a0315db7dad2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:36.201969 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4205f034-86be-4830-834b-a0315db7dad2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:36.203211 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3009067-d85a-4172-a66b-818df6b8b5d4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:36.203892 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4205f034-86be-4830-834b-a0315db7dad2 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:36.219682 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1346/2667] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:36.224608 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205f034-86be-4830-834b-a0315db7dad2 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:36.239996 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205f034-86be-4830-834b-a0315db7dad2 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:36.245764 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d08d56-8324-418d-9dcb-c4876be29ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:36.246456 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d08d56-8324-418d-9dcb-c4876be29ebe admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:36.246712 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d08d56-8324-418d-9dcb-c4876be29ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:36.247128 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d08d56-8324-418d-9dcb-c4876be29ebe admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:36.247961 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d08d56-8324-418d-9dcb-c4876be29ebe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:36.249722 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d08d56-8324-418d-9dcb-c4876be29ebe admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:36.261882 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1347/2668] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:21:36.266117 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d08d56-8324-418d-9dcb-c4876be29ebe admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:36.269252 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b3009067-d85a-4172-a66b-818df6b8b5d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45782c948da04daab84db1d8de609daf'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:36.277308 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d08d56-8324-418d-9dcb-c4876be29ebe admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:36.279243 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b3009067-d85a-4172-a66b-818df6b8b5d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45782c948da04daab84db1d8de609daf'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:36.280042 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b3009067-d85a-4172-a66b-818df6b8b5d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45782c948da04daab84db1d8de609daf'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:36.282962 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd1cfc-152f-48ea-afca-6bdbf7c3cc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:36.283563 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd1cfc-152f-48ea-afca-6bdbf7c3cc81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:36.283563 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd1cfc-152f-48ea-afca-6bdbf7c3cc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:36.283909 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd1cfc-152f-48ea-afca-6bdbf7c3cc81 admin admin] PATH_INFO: `/v3/projects/27e61e1ec0254aec8c10ccceb2ef3ecc/users/bc2802c848f24acaa8bd068cc8995915/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:36.294330 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b3009067-d85a-4172-a66b-818df6b8b5d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45782c948da04daab84db1d8de609daf'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:36.295053 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b3009067-d85a-4172-a66b-818df6b8b5d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45782c948da04daab84db1d8de609daf'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:36.313399 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cd1cfc-152f-48ea-afca-6bdbf7c3cc81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27e61e1ec0254aec8c10ccceb2ef3ecc, user_id=bc2802c848f24acaa8bd068cc8995915, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:36.320219 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1322/2669] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:36 2022] DELETE /identity/v3/users/45782c948da04daab84db1d8de609daf => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:36.320925 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cd1cfc-152f-48ea-afca-6bdbf7c3cc81 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:36.321940 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184be93d-1199-4bff-9e3b-7630d8f485b7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:36.322259 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184be93d-1199-4bff-9e3b-7630d8f485b7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:36.322506 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184be93d-1199-4bff-9e3b-7630d8f485b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:36.344143 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1348/2670] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:36 2022] PUT /identity/v3/projects/27e61e1ec0254aec8c10ccceb2ef3ecc/users/bc2802c848f24acaa8bd068cc8995915/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:36.349559 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-184be93d-1199-4bff-9e3b-7630d8f485b7 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:36.350976 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-184be93d-1199-4bff-9e3b-7630d8f485b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:36.355451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dfa46a-d7a7-43bc-ac29-896a8edd3553 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:36.355747 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dfa46a-d7a7-43bc-ac29-896a8edd3553 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:36.355945 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dfa46a-d7a7-43bc-ac29-896a8edd3553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:36.391801 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-d2dfa46a-d7a7-43bc-ac29-896a8edd3553 None None] MFA Rules not processed for user `bc2802c848f24acaa8bd068cc8995915`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:36.393080 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-d2dfa46a-d7a7-43bc-ac29-896a8edd3553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:36.422271 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:36.438292 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1323/2671] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:36.448356 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184be93d-1199-4bff-9e3b-7630d8f485b7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:36.449803 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-184be93d-1199-4bff-9e3b-7630d8f485b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:36.490321 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1349/2672] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:36.493579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dfa46a-d7a7-43bc-ac29-896a8edd3553 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:36.494991 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-d2dfa46a-d7a7-43bc-ac29-896a8edd3553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:36.510942 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dfa46a-d7a7-43bc-ac29-896a8edd3553 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:36.515573 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0c70d3-796d-44c3-93b9-4b82aee9e00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:36.516564 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c70d3-796d-44c3-93b9-4b82aee9e00f admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:36.516564 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c70d3-796d-44c3-93b9-4b82aee9e00f admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:36.517059 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c70d3-796d-44c3-93b9-4b82aee9e00f admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:36.517947 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0c70d3-796d-44c3-93b9-4b82aee9e00f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:36.520025 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0c70d3-796d-44c3-93b9-4b82aee9e00f admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:36.524060 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184be93d-1199-4bff-9e3b-7630d8f485b7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:36.527913 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47b50a2-08b5-4c79-8687-e18ac5047e88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:36.528576 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b50a2-08b5-4c79-8687-e18ac5047e88 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:36.528862 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b50a2-08b5-4c79-8687-e18ac5047e88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:36.529296 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b50a2-08b5-4c79-8687-e18ac5047e88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:36.541763 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47b50a2-08b5-4c79-8687-e18ac5047e88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:36.543368 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47b50a2-08b5-4c79-8687-e18ac5047e88 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:36.554994 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1324/2673] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:21:36.559822 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1350/2674] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:36 2022] POST /identity/v3/projects => generated 383 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:36.561513 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47b50a2-08b5-4c79-8687-e18ac5047e88 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:36.572540 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47b50a2-08b5-4c79-8687-e18ac5047e88 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:36.660551 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0a7a5-3a52-494e-b495-af33868dffcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:36.661874 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0a7a5-3a52-494e-b495-af33868dffcc admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:36.663781 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0a7a5-3a52-494e-b495-af33868dffcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:36.664181 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0a7a5-3a52-494e-b495-af33868dffcc admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:36.666657 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f0a7a5-3a52-494e-b495-af33868dffcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:36.671844 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f0a7a5-3a52-494e-b495-af33868dffcc admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:36.713122 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0c70d3-796d-44c3-93b9-4b82aee9e00f admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:36.713532 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1325/2675] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:36 2022] POST /identity/v3/users => generated 298 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:36.725415 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0c70d3-796d-44c3-93b9-4b82aee9e00f admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:36.731127 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47df8f05-0728-464d-b1cc-902c048bc041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:36.731767 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df8f05-0728-464d-b1cc-902c048bc041 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:36.732048 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df8f05-0728-464d-b1cc-902c048bc041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:36.732380 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df8f05-0728-464d-b1cc-902c048bc041 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:36.733172 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47df8f05-0728-464d-b1cc-902c048bc041 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:36.734786 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47df8f05-0728-464d-b1cc-902c048bc041 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:36.746510 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1351/2676] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:36.751039 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0a7a5-3a52-494e-b495-af33868dffcc admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:36.777216 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0a7a5-3a52-494e-b495-af33868dffcc admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:36.782468 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413afd9-3dee-4479-a287-c8725710f066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:36.783149 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413afd9-3dee-4479-a287-c8725710f066 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:36.783417 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413afd9-3dee-4479-a287-c8725710f066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:36.784048 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413afd9-3dee-4479-a287-c8725710f066 admin admin] PATH_INFO: `/v3/projects/918b349665a14147b975ea9e7d8498cd/users/0b56d604450a43bfbb67f892fb2fbd9b/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:36.808014 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5413afd9-3dee-4479-a287-c8725710f066 admin admin] RBAC: Authorizing `identity:create_grant(project_id=918b349665a14147b975ea9e7d8498cd, user_id=0b56d604450a43bfbb67f892fb2fbd9b, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:36.819536 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5413afd9-3dee-4479-a287-c8725710f066 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:36.847625 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1326/2677] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:36 2022] PUT /identity/v3/projects/918b349665a14147b975ea9e7d8498cd/users/0b56d604450a43bfbb67f892fb2fbd9b/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:36.852544 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47df8f05-0728-464d-b1cc-902c048bc041 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:36.865502 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47df8f05-0728-464d-b1cc-902c048bc041 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:36.914991 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:36.939331 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c043e9-35f0-4d53-813c-5fb4fc66592d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:36.940123 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c043e9-35f0-4d53-813c-5fb4fc66592d admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:36.940262 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c043e9-35f0-4d53-813c-5fb4fc66592d admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:36.940514 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c043e9-35f0-4d53-813c-5fb4fc66592d admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:36.941223 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c043e9-35f0-4d53-813c-5fb4fc66592d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:36.943588 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c043e9-35f0-4d53-813c-5fb4fc66592d admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:36.956921 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1352/2678] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:36 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:36.962351 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413afd9-3dee-4479-a287-c8725710f066 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:36.974979 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413afd9-3dee-4479-a287-c8725710f066 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:36.982204 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c043e9-35f0-4d53-813c-5fb4fc66592d admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:36.982774 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf47c9f-730d-4d6a-887a-e94c8696eea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:36.983667 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf47c9f-730d-4d6a-887a-e94c8696eea4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:36.983988 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf47c9f-730d-4d6a-887a-e94c8696eea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:36.984233 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf47c9f-730d-4d6a-887a-e94c8696eea4 admin admin] PATH_INFO: `/v3/projects/918b349665a14147b975ea9e7d8498cd/users/0b56d604450a43bfbb67f892fb2fbd9b/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:36.985943 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-d4c043e9-35f0-4d53-813c-5fb4fc66592d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:36.986755 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf47c9f-730d-4d6a-887a-e94c8696eea4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=918b349665a14147b975ea9e7d8498cd, user_id=0b56d604450a43bfbb67f892fb2fbd9b, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:36.990484 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf47c9f-730d-4d6a-887a-e94c8696eea4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:37.002477 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c043e9-35f0-4d53-813c-5fb4fc66592d admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:37.004262 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1327/2679] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:36 2022] PUT /identity/v3/projects/918b349665a14147b975ea9e7d8498cd/users/0b56d604450a43bfbb67f892fb2fbd9b/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:37.022625 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0fa41-9cfc-4739-8f07-1e923cf6a055 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:37.022913 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0fa41-9cfc-4739-8f07-1e923cf6a055 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:37.023605 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0fa41-9cfc-4739-8f07-1e923cf6a055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:37.053046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-09f0fa41-9cfc-4739-8f07-1e923cf6a055 None None] MFA Rules not processed for user `0b56d604450a43bfbb67f892fb2fbd9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:37.054334 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-09f0fa41-9cfc-4739-8f07-1e923cf6a055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:37.073900 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5420ed0-0b4d-46b4-b308-ebfad56e5349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:37.074651 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5420ed0-0b4d-46b4-b308-ebfad56e5349 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:37.074938 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5420ed0-0b4d-46b4-b308-ebfad56e5349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:37.075313 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5420ed0-0b4d-46b4-b308-ebfad56e5349 admin admin] PATH_INFO: `/v3/projects/eff6ded2c36846579af2f59ea9bfff8a` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:37.076689 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5420ed0-0b4d-46b4-b308-ebfad56e5349 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eff6ded2c36846579af2f59ea9bfff8a)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:37.078854 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5420ed0-0b4d-46b4-b308-ebfad56e5349 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:37.110655 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:37.139010 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-b5420ed0-0b4d-46b4-b308-ebfad56e5349 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eff6ded2c36846579af2f59ea9bfff8a', 'request_id': 'req-b5420ed0-0b4d-46b4-b308-ebfad56e5349'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:37.143908 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1328/2680] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:37 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:37.154634 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-b5420ed0-0b4d-46b4-b308-ebfad56e5349 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eff6ded2c36846579af2f59ea9bfff8a', 'request_id': 'req-b5420ed0-0b4d-46b4-b308-ebfad56e5349'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:37.157428 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0fa41-9cfc-4739-8f07-1e923cf6a055 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:37.159237 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-09f0fa41-9cfc-4739-8f07-1e923cf6a055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:37.163493 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-b5420ed0-0b4d-46b4-b308-ebfad56e5349 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eff6ded2c36846579af2f59ea9bfff8a', 'request_id': 'req-b5420ed0-0b4d-46b4-b308-ebfad56e5349'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:37.188220 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-b5420ed0-0b4d-46b4-b308-ebfad56e5349 admin admin] The token cache is being invalidate because project eff6ded2c36846579af2f59ea9bfff8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:37.189281 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-b5420ed0-0b4d-46b4-b308-ebfad56e5349 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:37.190192 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-b5420ed0-0b4d-46b4-b308-ebfad56e5349 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:37.192921 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1353/2681] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:36 2022] DELETE /identity/v3/projects/eff6ded2c36846579af2f59ea9bfff8a => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:37.197190 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5420ed0-0b4d-46b4-b308-ebfad56e5349 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:37.199790 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-b5420ed0-0b4d-46b4-b308-ebfad56e5349 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:37.237298 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0fa41-9cfc-4739-8f07-1e923cf6a055 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:37.319402 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544c993-16d0-42a9-8d27-61e49c3e7a7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:37.320041 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544c993-16d0-42a9-8d27-61e49c3e7a7f service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:37.320319 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544c993-16d0-42a9-8d27-61e49c3e7a7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:37.321307 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544c993-16d0-42a9-8d27-61e49c3e7a7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:37.323718 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b544c993-16d0-42a9-8d27-61e49c3e7a7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:37.326583 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5420ed0-0b4d-46b4-b308-ebfad56e5349 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:37.332052 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276a88d-c31e-40a8-9a73-0ac43adeeae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:37.333437 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276a88d-c31e-40a8-9a73-0ac43adeeae8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:37.333720 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276a88d-c31e-40a8-9a73-0ac43adeeae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:37.333998 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276a88d-c31e-40a8-9a73-0ac43adeeae8 admin admin] PATH_INFO: `/v3/users/36a7c310b6534ceca2d636aa722b977e` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:37.335187 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b276a88d-c31e-40a8-9a73-0ac43adeeae8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36a7c310b6534ceca2d636aa722b977e)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:37.337162 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b276a88d-c31e-40a8-9a73-0ac43adeeae8 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:37.344110 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:37.394933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-b276a88d-c31e-40a8-9a73-0ac43adeeae8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36a7c310b6534ceca2d636aa722b977e'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:37.396273 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-b276a88d-c31e-40a8-9a73-0ac43adeeae8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36a7c310b6534ceca2d636aa722b977e'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:37.404271 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b544c993-16d0-42a9-8d27-61e49c3e7a7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:37.406010 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b544c993-16d0-42a9-8d27-61e49c3e7a7f service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:37.409216 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-b276a88d-c31e-40a8-9a73-0ac43adeeae8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36a7c310b6534ceca2d636aa722b977e'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:37.423529 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-b276a88d-c31e-40a8-9a73-0ac43adeeae8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36a7c310b6534ceca2d636aa722b977e'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:37.424172 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-b276a88d-c31e-40a8-9a73-0ac43adeeae8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36a7c310b6534ceca2d636aa722b977e'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:37.424556 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1329/2682] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:37 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:21:37.442383 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1354/2683] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:37 2022] DELETE /identity/v3/users/36a7c310b6534ceca2d636aa722b977e => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:37.529697 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544c993-16d0-42a9-8d27-61e49c3e7a7f service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:37.531238 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b544c993-16d0-42a9-8d27-61e49c3e7a7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:37.544589 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544c993-16d0-42a9-8d27-61e49c3e7a7f service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:37.640001 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:37.661826 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19ef6a-ea50-4c18-996c-bc09a81c8ae1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:37.662473 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19ef6a-ea50-4c18-996c-bc09a81c8ae1 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:37.662762 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19ef6a-ea50-4c18-996c-bc09a81c8ae1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:37.663047 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19ef6a-ea50-4c18-996c-bc09a81c8ae1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:37.665653 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5c19ef6a-ea50-4c18-996c-bc09a81c8ae1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:37.734101 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c19ef6a-ea50-4c18-996c-bc09a81c8ae1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:37.735589 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c19ef6a-ea50-4c18-996c-bc09a81c8ae1 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:37.746490 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1330/2684] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:37.931147 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276a88d-c31e-40a8-9a73-0ac43adeeae8 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:37.935395 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-b276a88d-c31e-40a8-9a73-0ac43adeeae8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:38.083609 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276a88d-c31e-40a8-9a73-0ac43adeeae8 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:38.089293 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc1c28-37b2-484c-aacf-898327d74023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:38.089960 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc1c28-37b2-484c-aacf-898327d74023 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:38.090330 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc1c28-37b2-484c-aacf-898327d74023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:38.090738 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc1c28-37b2-484c-aacf-898327d74023 admin admin] PATH_INFO: `/v3/projects/9bdbd8cad10f4e04b7493f39235c815e` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:38.092001 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dc1c28-37b2-484c-aacf-898327d74023 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bdbd8cad10f4e04b7493f39235c815e)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:38.094190 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dc1c28-37b2-484c-aacf-898327d74023 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:38.180464 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19ef6a-ea50-4c18-996c-bc09a81c8ae1 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:38.182458 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5c19ef6a-ea50-4c18-996c-bc09a81c8ae1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:38.199190 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19ef6a-ea50-4c18-996c-bc09a81c8ae1 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:38.204057 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e2bb9a-1a69-411a-98a1-f12484b62b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:38.204893 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e2bb9a-1a69-411a-98a1-f12484b62b76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:38.205246 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e2bb9a-1a69-411a-98a1-f12484b62b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:38.205591 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e2bb9a-1a69-411a-98a1-f12484b62b76 admin admin] PATH_INFO: `/v3/users/bc2802c848f24acaa8bd068cc8995915` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:38.207026 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e2bb9a-1a69-411a-98a1-f12484b62b76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc2802c848f24acaa8bd068cc8995915)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:38.209769 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e2bb9a-1a69-411a-98a1-f12484b62b76 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:38.278377 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-41e2bb9a-1a69-411a-98a1-f12484b62b76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc2802c848f24acaa8bd068cc8995915'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:38.289885 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-41e2bb9a-1a69-411a-98a1-f12484b62b76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc2802c848f24acaa8bd068cc8995915'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:38.290930 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-41e2bb9a-1a69-411a-98a1-f12484b62b76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc2802c848f24acaa8bd068cc8995915'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:38.294248 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-13dc1c28-37b2-484c-aacf-898327d74023 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bdbd8cad10f4e04b7493f39235c815e', 'request_id': 'req-13dc1c28-37b2-484c-aacf-898327d74023'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:38.300882 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-41e2bb9a-1a69-411a-98a1-f12484b62b76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc2802c848f24acaa8bd068cc8995915'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:38.302605 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-41e2bb9a-1a69-411a-98a1-f12484b62b76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc2802c848f24acaa8bd068cc8995915'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:38.303074 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-13dc1c28-37b2-484c-aacf-898327d74023 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bdbd8cad10f4e04b7493f39235c815e', 'request_id': 'req-13dc1c28-37b2-484c-aacf-898327d74023'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:38.309613 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-13dc1c28-37b2-484c-aacf-898327d74023 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bdbd8cad10f4e04b7493f39235c815e', 'request_id': 'req-13dc1c28-37b2-484c-aacf-898327d74023'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:38.329901 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-13dc1c28-37b2-484c-aacf-898327d74023 admin admin] The token cache is being invalidate because project 9bdbd8cad10f4e04b7493f39235c815e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:38.330694 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-13dc1c28-37b2-484c-aacf-898327d74023 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:38.332160 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-13dc1c28-37b2-484c-aacf-898327d74023 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:38.334810 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1355/2685] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:37 2022] DELETE /identity/v3/projects/9bdbd8cad10f4e04b7493f39235c815e => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:38.335521 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1331/2686] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:38 2022] DELETE /identity/v3/users/bc2802c848f24acaa8bd068cc8995915 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:38.340898 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e7576f-8060-4720-9481-04c43992411a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:38.341045 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc1c28-37b2-484c-aacf-898327d74023 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:38.341380 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e7576f-8060-4720-9481-04c43992411a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:38.341768 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e7576f-8060-4720-9481-04c43992411a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:38.344206 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-13dc1c28-37b2-484c-aacf-898327d74023 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:38.365467 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:38.367909 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-63e7576f-8060-4720-9481-04c43992411a None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:38.369488 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-63e7576f-8060-4720-9481-04c43992411a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:38.425810 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc1c28-37b2-484c-aacf-898327d74023 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:38.462002 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1332/2687] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:38.473627 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e7576f-8060-4720-9481-04c43992411a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:38.475321 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-63e7576f-8060-4720-9481-04c43992411a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:38.494292 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:38.513246 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cfac84-2a11-4fb2-b05d-a0581e539948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:38.514465 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cfac84-2a11-4fb2-b05d-a0581e539948 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:38.514820 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cfac84-2a11-4fb2-b05d-a0581e539948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:38.515167 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cfac84-2a11-4fb2-b05d-a0581e539948 admin admin] PATH_INFO: `/v3/users/8faf04bf3b10453a975740b2aa080daf` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:38.516888 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cfac84-2a11-4fb2-b05d-a0581e539948 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8faf04bf3b10453a975740b2aa080daf)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:38.519168 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cfac84-2a11-4fb2-b05d-a0581e539948 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:38.547419 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e7576f-8060-4720-9481-04c43992411a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:38.552586 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ad67a-fdd9-4b62-802d-3775abbe07a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:38.553243 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ad67a-fdd9-4b62-802d-3775abbe07a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:38.553526 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ad67a-fdd9-4b62-802d-3775abbe07a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:38.553802 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ad67a-fdd9-4b62-802d-3775abbe07a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:38.564406 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ad67a-fdd9-4b62-802d-3775abbe07a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:38.565979 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ad67a-fdd9-4b62-802d-3775abbe07a2 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:38.573829 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-98cfac84-2a11-4fb2-b05d-a0581e539948 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8faf04bf3b10453a975740b2aa080daf'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:38.574569 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-98cfac84-2a11-4fb2-b05d-a0581e539948 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8faf04bf3b10453a975740b2aa080daf'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:38.575881 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1333/2688] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:38.576750 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ad67a-fdd9-4b62-802d-3775abbe07a2 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:38.578057 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ee4ad67a-fdd9-4b62-802d-3775abbe07a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:38.589452 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-98cfac84-2a11-4fb2-b05d-a0581e539948 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8faf04bf3b10453a975740b2aa080daf'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:38.591748 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ad67a-fdd9-4b62-802d-3775abbe07a2 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:38.597334 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac758725-c093-4b71-b802-982866975cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:38.597991 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac758725-c093-4b71-b802-982866975cb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:38.599153 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac758725-c093-4b71-b802-982866975cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:38.599153 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac758725-c093-4b71-b802-982866975cb7 admin admin] PATH_INFO: `/v3/users/86cef19aed2a45fb87c30dbd2d422dae` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:38.600122 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-98cfac84-2a11-4fb2-b05d-a0581e539948 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8faf04bf3b10453a975740b2aa080daf'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:38.600455 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac758725-c093-4b71-b802-982866975cb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86cef19aed2a45fb87c30dbd2d422dae)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:38.601499 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-98cfac84-2a11-4fb2-b05d-a0581e539948 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8faf04bf3b10453a975740b2aa080daf'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:38.602637 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac758725-c093-4b71-b802-982866975cb7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:38.618484 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1356/2689] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:38 2022] DELETE /identity/v3/users/8faf04bf3b10453a975740b2aa080daf => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:38.669641 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ac758725-c093-4b71-b802-982866975cb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86cef19aed2a45fb87c30dbd2d422dae'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:38.679368 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ac758725-c093-4b71-b802-982866975cb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86cef19aed2a45fb87c30dbd2d422dae'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:38.680565 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ac758725-c093-4b71-b802-982866975cb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86cef19aed2a45fb87c30dbd2d422dae'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:38.690861 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ac758725-c093-4b71-b802-982866975cb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86cef19aed2a45fb87c30dbd2d422dae'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:38.692033 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ac758725-c093-4b71-b802-982866975cb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86cef19aed2a45fb87c30dbd2d422dae'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:38.736855 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1334/2690] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:38 2022] DELETE /identity/v3/users/86cef19aed2a45fb87c30dbd2d422dae => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:38.977273 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cfac84-2a11-4fb2-b05d-a0581e539948 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:38.978767 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-98cfac84-2a11-4fb2-b05d-a0581e539948 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:39.024067 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac758725-c093-4b71-b802-982866975cb7 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:39.025908 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ac758725-c093-4b71-b802-982866975cb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:39.035259 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:39.060532 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cfac84-2a11-4fb2-b05d-a0581e539948 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:39.111533 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac758725-c093-4b71-b802-982866975cb7 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:39.116531 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f394e41-11cc-40c9-81ca-25fe3642f64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:39.117279 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f394e41-11cc-40c9-81ca-25fe3642f64c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:39.117461 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f394e41-11cc-40c9-81ca-25fe3642f64c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:39.117750 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f394e41-11cc-40c9-81ca-25fe3642f64c admin admin] PATH_INFO: `/v3/projects/27e61e1ec0254aec8c10ccceb2ef3ecc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:39.118903 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f394e41-11cc-40c9-81ca-25fe3642f64c admin admin] RBAC: Authorizing `identity:delete_project(project_id=27e61e1ec0254aec8c10ccceb2ef3ecc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:39.120822 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f394e41-11cc-40c9-81ca-25fe3642f64c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:39.141404 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d1066d-16d9-4830-bb68-09632a015288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:39.142060 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1066d-16d9-4830-bb68-09632a015288 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:39.142344 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1066d-16d9-4830-bb68-09632a015288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:39.142618 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1066d-16d9-4830-bb68-09632a015288 admin admin] PATH_INFO: `/v3/projects/10c3c2edbbd2452bb94379fb9f28f871` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:39.143776 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d1066d-16d9-4830-bb68-09632a015288 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10c3c2edbbd2452bb94379fb9f28f871)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:39.146316 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d1066d-16d9-4830-bb68-09632a015288 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:39.156658 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:39.171926 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8f394e41-11cc-40c9-81ca-25fe3642f64c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27e61e1ec0254aec8c10ccceb2ef3ecc', 'request_id': 'req-8f394e41-11cc-40c9-81ca-25fe3642f64c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:39.178908 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8f394e41-11cc-40c9-81ca-25fe3642f64c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27e61e1ec0254aec8c10ccceb2ef3ecc', 'request_id': 'req-8f394e41-11cc-40c9-81ca-25fe3642f64c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:39.186194 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8f394e41-11cc-40c9-81ca-25fe3642f64c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27e61e1ec0254aec8c10ccceb2ef3ecc', 'request_id': 'req-8f394e41-11cc-40c9-81ca-25fe3642f64c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:39.196072 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-25d1066d-16d9-4830-bb68-09632a015288 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10c3c2edbbd2452bb94379fb9f28f871', 'request_id': 'req-25d1066d-16d9-4830-bb68-09632a015288'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:39.197699 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8f394e41-11cc-40c9-81ca-25fe3642f64c admin admin] The token cache is being invalidate because project 27e61e1ec0254aec8c10ccceb2ef3ecc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:39.197990 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8f394e41-11cc-40c9-81ca-25fe3642f64c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:39.198629 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8f394e41-11cc-40c9-81ca-25fe3642f64c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:39.200164 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1335/2691] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:39 2022] DELETE /identity/v3/projects/27e61e1ec0254aec8c10ccceb2ef3ecc => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:39.200965 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f394e41-11cc-40c9-81ca-25fe3642f64c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:39.202256 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8f394e41-11cc-40c9-81ca-25fe3642f64c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:39.205399 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-25d1066d-16d9-4830-bb68-09632a015288 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10c3c2edbbd2452bb94379fb9f28f871', 'request_id': 'req-25d1066d-16d9-4830-bb68-09632a015288'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:39.212797 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-25d1066d-16d9-4830-bb68-09632a015288 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10c3c2edbbd2452bb94379fb9f28f871', 'request_id': 'req-25d1066d-16d9-4830-bb68-09632a015288'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:39.226589 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-25d1066d-16d9-4830-bb68-09632a015288 admin admin] The token cache is being invalidate because project 10c3c2edbbd2452bb94379fb9f28f871 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:39.226882 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-25d1066d-16d9-4830-bb68-09632a015288 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:39.227513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-25d1066d-16d9-4830-bb68-09632a015288 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:39.229896 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1357/2692] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:38 2022] DELETE /identity/v3/projects/10c3c2edbbd2452bb94379fb9f28f871 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:39.230740 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d1066d-16d9-4830-bb68-09632a015288 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:39.232050 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-25d1066d-16d9-4830-bb68-09632a015288 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:39.270735 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f394e41-11cc-40c9-81ca-25fe3642f64c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:39.297760 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d1066d-16d9-4830-bb68-09632a015288 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:39.302416 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf635f0-6242-47c2-aa52-eafc215e3b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:39.303058 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf635f0-6242-47c2-aa52-eafc215e3b3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:39.303337 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf635f0-6242-47c2-aa52-eafc215e3b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:39.303609 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf635f0-6242-47c2-aa52-eafc215e3b3c admin admin] PATH_INFO: `/v3/users/0b56d604450a43bfbb67f892fb2fbd9b` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:39.304802 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf635f0-6242-47c2-aa52-eafc215e3b3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b56d604450a43bfbb67f892fb2fbd9b)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:39.306689 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf635f0-6242-47c2-aa52-eafc215e3b3c admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:39.327246 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daec9ae-39a9-4a3e-89a4-8abad92c0fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:39.327889 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daec9ae-39a9-4a3e-89a4-8abad92c0fec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:39.328171 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daec9ae-39a9-4a3e-89a4-8abad92c0fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:39.328449 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daec9ae-39a9-4a3e-89a4-8abad92c0fec admin admin] PATH_INFO: `/v3/projects/d11b6361110344b0a67b6b73ccc0c2f4` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:39.328793 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:39.329799 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daec9ae-39a9-4a3e-89a4-8abad92c0fec admin admin] RBAC: Authorizing `identity:delete_project(project_id=d11b6361110344b0a67b6b73ccc0c2f4)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:39.331862 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daec9ae-39a9-4a3e-89a4-8abad92c0fec admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:39.354589 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-ccf635f0-6242-47c2-aa52-eafc215e3b3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b56d604450a43bfbb67f892fb2fbd9b'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:39.355184 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-ccf635f0-6242-47c2-aa52-eafc215e3b3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b56d604450a43bfbb67f892fb2fbd9b'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:39.368183 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-ccf635f0-6242-47c2-aa52-eafc215e3b3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b56d604450a43bfbb67f892fb2fbd9b'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:39.375612 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-ccf635f0-6242-47c2-aa52-eafc215e3b3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b56d604450a43bfbb67f892fb2fbd9b'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:39.376302 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-ccf635f0-6242-47c2-aa52-eafc215e3b3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b56d604450a43bfbb67f892fb2fbd9b'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:39.380893 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5daec9ae-39a9-4a3e-89a4-8abad92c0fec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd11b6361110344b0a67b6b73ccc0c2f4', 'request_id': 'req-5daec9ae-39a9-4a3e-89a4-8abad92c0fec'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:39.389150 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5daec9ae-39a9-4a3e-89a4-8abad92c0fec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd11b6361110344b0a67b6b73ccc0c2f4', 'request_id': 'req-5daec9ae-39a9-4a3e-89a4-8abad92c0fec'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:39.392820 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1358/2693] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:39 2022] DELETE /identity/v3/users/0b56d604450a43bfbb67f892fb2fbd9b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:39.392820 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8b2a6-22d7-41a9-9ea4-e9add3ef9a27 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:39.392820 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8b2a6-22d7-41a9-9ea4-e9add3ef9a27 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:39.392820 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8b2a6-22d7-41a9-9ea4-e9add3ef9a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:39.396702 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5daec9ae-39a9-4a3e-89a4-8abad92c0fec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd11b6361110344b0a67b6b73ccc0c2f4', 'request_id': 'req-5daec9ae-39a9-4a3e-89a4-8abad92c0fec'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:39.403218 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:39.412652 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5daec9ae-39a9-4a3e-89a4-8abad92c0fec admin admin] The token cache is being invalidate because project d11b6361110344b0a67b6b73ccc0c2f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:39.413120 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5daec9ae-39a9-4a3e-89a4-8abad92c0fec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:39.414085 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5daec9ae-39a9-4a3e-89a4-8abad92c0fec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:39.415900 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1336/2694] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:39 2022] DELETE /identity/v3/projects/d11b6361110344b0a67b6b73ccc0c2f4 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:39.416583 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-63e8b2a6-22d7-41a9-9ea4-e9add3ef9a27 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:39.417802 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-63e8b2a6-22d7-41a9-9ea4-e9add3ef9a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:39.419626 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daec9ae-39a9-4a3e-89a4-8abad92c0fec admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:39.421007 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5daec9ae-39a9-4a3e-89a4-8abad92c0fec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:39.517374 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1359/2695] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:39.525948 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94562a37-496f-4e60-be0a-9a5207c1f06d None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:39.526269 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94562a37-496f-4e60-be0a-9a5207c1f06d None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:39.526550 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94562a37-496f-4e60-be0a-9a5207c1f06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:39.526836 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daec9ae-39a9-4a3e-89a4-8abad92c0fec admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:39.531843 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e88a4-145b-4c07-8457-fb18da6b2bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:39.532538 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e88a4-145b-4c07-8457-fb18da6b2bc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:39.533368 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e88a4-145b-4c07-8457-fb18da6b2bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:39.533655 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e88a4-145b-4c07-8457-fb18da6b2bc5 admin admin] PATH_INFO: `/v3/users/fd6769ee862844e19b53ff6855acd760` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:39.534921 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3e88a4-145b-4c07-8457-fb18da6b2bc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd6769ee862844e19b53ff6855acd760)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:39.536941 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3e88a4-145b-4c07-8457-fb18da6b2bc5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:39.554609 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-94562a37-496f-4e60-be0a-9a5207c1f06d None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:39.555840 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-94562a37-496f-4e60-be0a-9a5207c1f06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:39.565155 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1360/2696] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:21:39.569381 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94562a37-496f-4e60-be0a-9a5207c1f06d None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:39.580640 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94562a37-496f-4e60-be0a-9a5207c1f06d None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:39.586240 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3909e7-7c07-4164-ad97-8d9a6cadacc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:39.586866 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3909e7-7c07-4164-ad97-8d9a6cadacc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:39.587152 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3909e7-7c07-4164-ad97-8d9a6cadacc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:39.587505 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3909e7-7c07-4164-ad97-8d9a6cadacc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:39.588366 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3909e7-7c07-4164-ad97-8d9a6cadacc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:21:39.588681 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3909e7-7c07-4164-ad97-8d9a6cadacc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:39.590293 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3909e7-7c07-4164-ad97-8d9a6cadacc9 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:39.597775 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-db3e88a4-145b-4c07-8457-fb18da6b2bc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd6769ee862844e19b53ff6855acd760'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:39.604459 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1361/2697] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:21:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:21:39.608781 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3909e7-7c07-4164-ad97-8d9a6cadacc9 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:39.610892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-db3e88a4-145b-4c07-8457-fb18da6b2bc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd6769ee862844e19b53ff6855acd760'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:39.611831 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-db3e88a4-145b-4c07-8457-fb18da6b2bc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd6769ee862844e19b53ff6855acd760'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:39.619518 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3909e7-7c07-4164-ad97-8d9a6cadacc9 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:39.624314 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-db3e88a4-145b-4c07-8457-fb18da6b2bc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd6769ee862844e19b53ff6855acd760'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:39.625099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-db3e88a4-145b-4c07-8457-fb18da6b2bc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd6769ee862844e19b53ff6855acd760'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:39.625238 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602254b8-462e-49ed-b292-011e6a91d999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:39.625969 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602254b8-462e-49ed-b292-011e6a91d999 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:39.626311 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602254b8-462e-49ed-b292-011e6a91d999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:39.626641 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602254b8-462e-49ed-b292-011e6a91d999 admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:39.627245 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602254b8-462e-49ed-b292-011e6a91d999 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:39.629217 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602254b8-462e-49ed-b292-011e6a91d999 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:39.645675 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1337/2698] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:39 2022] DELETE /identity/v3/users/fd6769ee862844e19b53ff6855acd760 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:39.664140 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1362/2699] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:39 2022] POST /identity/v3/projects => generated 401 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:21:39.668256 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e88a4-145b-4c07-8457-fb18da6b2bc5 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:39.669781 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-db3e88a4-145b-4c07-8457-fb18da6b2bc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:39.740667 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e88a4-145b-4c07-8457-fb18da6b2bc5 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:39.750620 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602254b8-462e-49ed-b292-011e6a91d999 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:39.752363 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-602254b8-462e-49ed-b292-011e6a91d999 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:39.763227 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:39.818870 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ac1204-d213-4010-ad66-db0383760a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:39.819520 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac1204-d213-4010-ad66-db0383760a22 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:39.822896 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac1204-d213-4010-ad66-db0383760a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:39.823199 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac1204-d213-4010-ad66-db0383760a22 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:39.823915 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ac1204-d213-4010-ad66-db0383760a22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:39.826049 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ac1204-d213-4010-ad66-db0383760a22 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:39.847081 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:39.850803 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1338/2700] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:39 2022] POST /identity/v3/users => generated 301 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:21:39.852004 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602254b8-462e-49ed-b292-011e6a91d999 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:39.855347 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ac1204-d213-4010-ad66-db0383760a22 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:39.858535 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b121c9-b0c8-4942-aacb-f8cf7ec18660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:39.859287 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b121c9-b0c8-4942-aacb-f8cf7ec18660 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:39.859916 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b121c9-b0c8-4942-aacb-f8cf7ec18660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:39.860268 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b121c9-b0c8-4942-aacb-f8cf7ec18660 admin admin] PATH_INFO: `/v3/projects/918b349665a14147b975ea9e7d8498cd` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:39.861725 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b121c9-b0c8-4942-aacb-f8cf7ec18660 admin admin] RBAC: Authorizing `identity:delete_project(project_id=918b349665a14147b975ea9e7d8498cd)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:39.864247 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b121c9-b0c8-4942-aacb-f8cf7ec18660 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:39.868385 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ac1204-d213-4010-ad66-db0383760a22 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:39.874852 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c682e3-b770-4820-aba1-ba5772045ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:39.875536 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c682e3-b770-4820-aba1-ba5772045ae8 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:39.875828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c682e3-b770-4820-aba1-ba5772045ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:39.876130 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c682e3-b770-4820-aba1-ba5772045ae8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:39.876992 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c682e3-b770-4820-aba1-ba5772045ae8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:39.878723 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c682e3-b770-4820-aba1-ba5772045ae8 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:39.894681 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1339/2701] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:21:39.900256 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c682e3-b770-4820-aba1-ba5772045ae8 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:39.913700 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c682e3-b770-4820-aba1-ba5772045ae8 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:39.927416 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-29b121c9-b0c8-4942-aacb-f8cf7ec18660 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '918b349665a14147b975ea9e7d8498cd', 'request_id': 'req-29b121c9-b0c8-4942-aacb-f8cf7ec18660'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:39.936136 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-29b121c9-b0c8-4942-aacb-f8cf7ec18660 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '918b349665a14147b975ea9e7d8498cd', 'request_id': 'req-29b121c9-b0c8-4942-aacb-f8cf7ec18660'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:39.942695 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-29b121c9-b0c8-4942-aacb-f8cf7ec18660 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '918b349665a14147b975ea9e7d8498cd', 'request_id': 'req-29b121c9-b0c8-4942-aacb-f8cf7ec18660'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:39.956728 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-29b121c9-b0c8-4942-aacb-f8cf7ec18660 admin admin] The token cache is being invalidate because project 918b349665a14147b975ea9e7d8498cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:39.957101 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-29b121c9-b0c8-4942-aacb-f8cf7ec18660 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:39.958396 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-29b121c9-b0c8-4942-aacb-f8cf7ec18660 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:39.960411 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1363/2702] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:39 2022] DELETE /identity/v3/projects/918b349665a14147b975ea9e7d8498cd => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:39.964800 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b121c9-b0c8-4942-aacb-f8cf7ec18660 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:39.966130 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-29b121c9-b0c8-4942-aacb-f8cf7ec18660 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:39.979728 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba8944c-e1fe-4989-9824-0cb96954469e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:39.980351 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8944c-e1fe-4989-9824-0cb96954469e admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:39.980669 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8944c-e1fe-4989-9824-0cb96954469e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:39.980954 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8944c-e1fe-4989-9824-0cb96954469e admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:39.981680 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba8944c-e1fe-4989-9824-0cb96954469e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:39.983273 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba8944c-e1fe-4989-9824-0cb96954469e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:39.993342 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1340/2703] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:39 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:39.997659 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b2c19-0f12-4777-8fde-3387c3091ada None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:39.997953 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b2c19-0f12-4777-8fde-3387c3091ada None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:39.998238 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b2c19-0f12-4777-8fde-3387c3091ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:40.017846 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-887b2c19-0f12-4777-8fde-3387c3091ada None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:40.019061 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-887b2c19-0f12-4777-8fde-3387c3091ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:40.027751 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1341/2704] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:40.028612 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887b2c19-0f12-4777-8fde-3387c3091ada None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:40.035090 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b121c9-b0c8-4942-aacb-f8cf7ec18660 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:40.036749 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-887b2c19-0f12-4777-8fde-3387c3091ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:40.050601 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887b2c19-0f12-4777-8fde-3387c3091ada None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:40.054228 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9248a2-3d31-4bf1-a31a-90ec03391c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:40.054922 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9248a2-3d31-4bf1-a31a-90ec03391c98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:40.055206 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9248a2-3d31-4bf1-a31a-90ec03391c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:40.055485 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9248a2-3d31-4bf1-a31a-90ec03391c98 admin admin] PATH_INFO: `/v3/projects/0808b80655a349a792b3fb4c96cab884` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:40.055860 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f23909-300b-4951-9e8c-758f0b1d93c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:40.056591 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f23909-300b-4951-9e8c-758f0b1d93c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:40.056828 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9248a2-3d31-4bf1-a31a-90ec03391c98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0808b80655a349a792b3fb4c96cab884)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:40.057044 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f23909-300b-4951-9e8c-758f0b1d93c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:40.057300 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f23909-300b-4951-9e8c-758f0b1d93c2 admin admin] PATH_INFO: `/v3/projects/95b5987a051942b196b5d956cc2fd5ce/users/820e52f08ec84493b68726505679752f/roles/926b9f36c24847c582c84af904250abc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:40.059263 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9248a2-3d31-4bf1-a31a-90ec03391c98 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:40.075808 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f23909-300b-4951-9e8c-758f0b1d93c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95b5987a051942b196b5d956cc2fd5ce, user_id=820e52f08ec84493b68726505679752f, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:40.078594 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f23909-300b-4951-9e8c-758f0b1d93c2 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:40.095359 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1342/2705] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:40 2022] PUT /identity/v3/projects/95b5987a051942b196b5d956cc2fd5ce/users/820e52f08ec84493b68726505679752f/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:40.096659 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932c305-1e0c-4a62-959c-5c39e7c2935a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:40.097095 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932c305-1e0c-4a62-959c-5c39e7c2935a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:40.097375 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932c305-1e0c-4a62-959c-5c39e7c2935a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:40.117974 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5932c305-1e0c-4a62-959c-5c39e7c2935a None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:40.119107 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5932c305-1e0c-4a62-959c-5c39e7c2935a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:40.123044 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-7d9248a2-3d31-4bf1-a31a-90ec03391c98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0808b80655a349a792b3fb4c96cab884', 'request_id': 'req-7d9248a2-3d31-4bf1-a31a-90ec03391c98'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:40.131247 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-7d9248a2-3d31-4bf1-a31a-90ec03391c98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0808b80655a349a792b3fb4c96cab884', 'request_id': 'req-7d9248a2-3d31-4bf1-a31a-90ec03391c98'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:40.137514 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-7d9248a2-3d31-4bf1-a31a-90ec03391c98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0808b80655a349a792b3fb4c96cab884', 'request_id': 'req-7d9248a2-3d31-4bf1-a31a-90ec03391c98'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:40.150204 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-7d9248a2-3d31-4bf1-a31a-90ec03391c98 admin admin] The token cache is being invalidate because project 0808b80655a349a792b3fb4c96cab884 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:40.150825 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-7d9248a2-3d31-4bf1-a31a-90ec03391c98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:40.151348 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-7d9248a2-3d31-4bf1-a31a-90ec03391c98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:40.152817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1364/2706] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:39 2022] DELETE /identity/v3/projects/0808b80655a349a792b3fb4c96cab884 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:40.154074 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf225f-4ce6-4b0f-9a2b-19cdcc0e1729 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:40.155062 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf225f-4ce6-4b0f-9a2b-19cdcc0e1729 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:40.155062 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf225f-4ce6-4b0f-9a2b-19cdcc0e1729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:40.184521 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1343/2707] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:40.189851 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-57bf225f-4ce6-4b0f-9a2b-19cdcc0e1729 None None] MFA Rules not processed for user `820e52f08ec84493b68726505679752f`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:40.190098 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5932c305-1e0c-4a62-959c-5c39e7c2935a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:40.191089 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-57bf225f-4ce6-4b0f-9a2b-19cdcc0e1729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:40.197964 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:40.200260 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5932c305-1e0c-4a62-959c-5c39e7c2935a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:40.205661 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e02da3-3d85-43b9-a5e2-3e1321d16276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:40.206237 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e02da3-3d85-43b9-a5e2-3e1321d16276 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:40.206530 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e02da3-3d85-43b9-a5e2-3e1321d16276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:40.206881 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e02da3-3d85-43b9-a5e2-3e1321d16276 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:40.207663 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e02da3-3d85-43b9-a5e2-3e1321d16276 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:21:40.207987 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e02da3-3d85-43b9-a5e2-3e1321d16276 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:40.210032 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e02da3-3d85-43b9-a5e2-3e1321d16276 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:40.212060 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:40.225462 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1344/2708] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:21:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:21:40.229858 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e02da3-3d85-43b9-a5e2-3e1321d16276 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:40.240278 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e02da3-3d85-43b9-a5e2-3e1321d16276 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:40.245322 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3fda59-2658-4528-84cb-dfe50d018bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:40.245963 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3fda59-2658-4528-84cb-dfe50d018bdf admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:40.246240 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3fda59-2658-4528-84cb-dfe50d018bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:40.246523 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3fda59-2658-4528-84cb-dfe50d018bdf admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:40.247208 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3fda59-2658-4528-84cb-dfe50d018bdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:40.249266 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3fda59-2658-4528-84cb-dfe50d018bdf admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:40.279698 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1345/2709] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:40 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:21:40.281056 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1365/2710] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:40 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:40.285688 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3fda59-2658-4528-84cb-dfe50d018bdf admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:40.285968 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf225f-4ce6-4b0f-9a2b-19cdcc0e1729 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:40.288211 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-57bf225f-4ce6-4b0f-9a2b-19cdcc0e1729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:40.303643 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf225f-4ce6-4b0f-9a2b-19cdcc0e1729 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:40.362267 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3fda59-2658-4528-84cb-dfe50d018bdf admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:40.368365 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710b162-fea4-4117-b74c-7e4223b58b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:40.369171 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710b162-fea4-4117-b74c-7e4223b58b92 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:40.369462 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710b162-fea4-4117-b74c-7e4223b58b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:40.369740 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710b162-fea4-4117-b74c-7e4223b58b92 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:40.370582 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e710b162-fea4-4117-b74c-7e4223b58b92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:40.371651 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f43c12b-5334-420a-bfb8-be36e1f35faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:40.372302 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f43c12b-5334-420a-bfb8-be36e1f35faa admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:40.372691 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e710b162-fea4-4117-b74c-7e4223b58b92 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:40.372801 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f43c12b-5334-420a-bfb8-be36e1f35faa admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:40.372976 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f43c12b-5334-420a-bfb8-be36e1f35faa admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:40.373665 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f43c12b-5334-420a-bfb8-be36e1f35faa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:40.375592 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f43c12b-5334-420a-bfb8-be36e1f35faa admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:40.400637 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1346/2711] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:40 2022] POST /identity/v3/users => generated 288 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:40.404998 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710b162-fea4-4117-b74c-7e4223b58b92 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:40.413808 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1366/2712] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:40 2022] POST /identity/v3/projects => generated 401 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:40.414912 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f43c12b-5334-420a-bfb8-be36e1f35faa admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:40.419943 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710b162-fea4-4117-b74c-7e4223b58b92 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:40.484197 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f43c12b-5334-420a-bfb8-be36e1f35faa admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:40.484424 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9584f556-4438-4572-aa68-3e9da13f6ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:40.484887 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584f556-4438-4572-aa68-3e9da13f6ae6 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:40.485176 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584f556-4438-4572-aa68-3e9da13f6ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:40.485451 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584f556-4438-4572-aa68-3e9da13f6ae6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:40.486144 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9584f556-4438-4572-aa68-3e9da13f6ae6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:40.487729 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9584f556-4438-4572-aa68-3e9da13f6ae6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:40.489104 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abeea00-e331-4367-9278-0bc62df1160c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:40.489743 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abeea00-e331-4367-9278-0bc62df1160c admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:40.490013 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abeea00-e331-4367-9278-0bc62df1160c admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:40.490291 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abeea00-e331-4367-9278-0bc62df1160c admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:40.491007 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abeea00-e331-4367-9278-0bc62df1160c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:40.492931 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abeea00-e331-4367-9278-0bc62df1160c admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:40.498029 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1347/2713] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:40.501829 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9584f556-4438-4572-aa68-3e9da13f6ae6 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:40.512391 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9584f556-4438-4572-aa68-3e9da13f6ae6 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:40.517127 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1367/2714] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:40 2022] POST /identity/v3/users => generated 307 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:21:40.517518 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b867be2f-d676-4637-adb6-6f3a82d57b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:40.518188 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867be2f-d676-4637-adb6-6f3a82d57b15 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:40.518471 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867be2f-d676-4637-adb6-6f3a82d57b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:40.519003 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867be2f-d676-4637-adb6-6f3a82d57b15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:40.519433 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b867be2f-d676-4637-adb6-6f3a82d57b15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:40.521781 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abeea00-e331-4367-9278-0bc62df1160c admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:40.521933 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b867be2f-d676-4637-adb6-6f3a82d57b15 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:40.532898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abeea00-e331-4367-9278-0bc62df1160c admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:40.538475 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1348/2715] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:21:40.538615 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8865b97-5604-443e-b34b-f181713d0479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:40.538615 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8865b97-5604-443e-b34b-f181713d0479 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:40.538816 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8865b97-5604-443e-b34b-f181713d0479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:40.538914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8865b97-5604-443e-b34b-f181713d0479 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:40.539546 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b867be2f-d676-4637-adb6-6f3a82d57b15 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:40.539661 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8865b97-5604-443e-b34b-f181713d0479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:40.541322 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8865b97-5604-443e-b34b-f181713d0479 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:40.550283 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b867be2f-d676-4637-adb6-6f3a82d57b15 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:40.554567 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1368/2716] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:40.556685 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112fd16e-920d-43e5-bca2-cc59b39dc524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:40.557437 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112fd16e-920d-43e5-bca2-cc59b39dc524 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:40.557732 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112fd16e-920d-43e5-bca2-cc59b39dc524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:40.558010 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112fd16e-920d-43e5-bca2-cc59b39dc524 admin admin] PATH_INFO: `/v3/projects/3de24e2ea8e24ff2a789cccf73440188/users/7f89506c03d34f9faecb80c073fb0add/roles/926b9f36c24847c582c84af904250abc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:40.558829 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8865b97-5604-443e-b34b-f181713d0479 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:40.570062 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8865b97-5604-443e-b34b-f181713d0479 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:40.575063 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3b5d6-cf59-48ae-b867-29030024b835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:40.575730 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3b5d6-cf59-48ae-b867-29030024b835 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:40.576012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3b5d6-cf59-48ae-b867-29030024b835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:40.576422 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3b5d6-cf59-48ae-b867-29030024b835 admin admin] PATH_INFO: `/v3/projects/3554a8e300da45939f332d10897002ce/users/d6b22b0be97947d1b342cd25e65e32ab/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:40.576558 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112fd16e-920d-43e5-bca2-cc59b39dc524 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3de24e2ea8e24ff2a789cccf73440188, user_id=7f89506c03d34f9faecb80c073fb0add, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:40.579305 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112fd16e-920d-43e5-bca2-cc59b39dc524 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:40.585079 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:40.593408 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d3b5d6-cf59-48ae-b867-29030024b835 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3554a8e300da45939f332d10897002ce, user_id=d6b22b0be97947d1b342cd25e65e32ab, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:40.596236 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d3b5d6-cf59-48ae-b867-29030024b835 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:40.604585 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1349/2717] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:40 2022] PUT /identity/v3/projects/3de24e2ea8e24ff2a789cccf73440188/users/7f89506c03d34f9faecb80c073fb0add/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:40.615393 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c818a46-2e5a-44da-821a-e48263febab2 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:40.615690 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c818a46-2e5a-44da-821a-e48263febab2 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:40.615826 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1369/2718] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:40 2022] PUT /identity/v3/projects/3554a8e300da45939f332d10897002ce/users/d6b22b0be97947d1b342cd25e65e32ab/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:40.615982 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c818a46-2e5a-44da-821a-e48263febab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:40.619971 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3b5d6-cf59-48ae-b867-29030024b835 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:40.649717 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-3c818a46-2e5a-44da-821a-e48263febab2 None None] MFA Rules not processed for user `7f89506c03d34f9faecb80c073fb0add`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:40.650949 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3c818a46-2e5a-44da-821a-e48263febab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:40.661705 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:40.692372 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3b5d6-cf59-48ae-b867-29030024b835 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:40.699255 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236e3503-2c06-4570-ab12-91ab75895544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:40.699891 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e3503-2c06-4570-ab12-91ab75895544 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:40.700197 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e3503-2c06-4570-ab12-91ab75895544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:40.700451 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e3503-2c06-4570-ab12-91ab75895544 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:40.701178 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236e3503-2c06-4570-ab12-91ab75895544 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:40.702774 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236e3503-2c06-4570-ab12-91ab75895544 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:40.717750 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1370/2719] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:40.721880 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236e3503-2c06-4570-ab12-91ab75895544 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:40.734157 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1350/2720] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:21:40.734446 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236e3503-2c06-4570-ab12-91ab75895544 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:40.737266 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd1f35-cd05-4a86-a760-77ccdfef0398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:40.737937 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd1f35-cd05-4a86-a760-77ccdfef0398 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:40.738217 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd1f35-cd05-4a86-a760-77ccdfef0398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:40.738494 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd1f35-cd05-4a86-a760-77ccdfef0398 admin admin] PATH_INFO: `/v3/projects/3554a8e300da45939f332d10897002ce/users/d6b22b0be97947d1b342cd25e65e32ab/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:40.740663 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fd1f35-cd05-4a86-a760-77ccdfef0398 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3554a8e300da45939f332d10897002ce, user_id=d6b22b0be97947d1b342cd25e65e32ab, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:40.741675 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c818a46-2e5a-44da-821a-e48263febab2 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:40.743041 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3c818a46-2e5a-44da-821a-e48263febab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:40.746914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fd1f35-cd05-4a86-a760-77ccdfef0398 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:40.763359 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1371/2721] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:40 2022] PUT /identity/v3/projects/3554a8e300da45939f332d10897002ce/users/d6b22b0be97947d1b342cd25e65e32ab/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:40.774496 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381bd41c-3ef6-4885-bbe3-926256db4d23 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:40.774816 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381bd41c-3ef6-4885-bbe3-926256db4d23 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:40.775100 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381bd41c-3ef6-4885-bbe3-926256db4d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:40.804585 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-381bd41c-3ef6-4885-bbe3-926256db4d23 None None] MFA Rules not processed for user `d6b22b0be97947d1b342cd25e65e32ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:40.805841 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-381bd41c-3ef6-4885-bbe3-926256db4d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:40.822537 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c818a46-2e5a-44da-821a-e48263febab2 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:40.882325 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00018110-700a-4886-8109-b312f6f8ef02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:40.886170 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00018110-700a-4886-8109-b312f6f8ef02 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:40.886170 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00018110-700a-4886-8109-b312f6f8ef02 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:40.886170 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00018110-700a-4886-8109-b312f6f8ef02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:40.891155 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1372/2722] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:40 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:40.946314 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00018110-700a-4886-8109-b312f6f8ef02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:40.947998 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00018110-700a-4886-8109-b312f6f8ef02 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:40.953714 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381bd41c-3ef6-4885-bbe3-926256db4d23 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:40.955330 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-381bd41c-3ef6-4885-bbe3-926256db4d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:40.958129 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1351/2723] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:21:40.968036 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381bd41c-3ef6-4885-bbe3-926256db4d23 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:40.972811 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc22c1-063c-4387-b45f-b9754248773c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:40.973972 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc22c1-063c-4387-b45f-b9754248773c service cinder] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:40.974254 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc22c1-063c-4387-b45f-b9754248773c service cinder] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:40.974527 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc22c1-063c-4387-b45f-b9754248773c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:40.984340 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc22c1-063c-4387-b45f-b9754248773c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:40.985789 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc22c1-063c-4387-b45f-b9754248773c service cinder] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:40.996909 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1373/2724] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:40 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:21:41.076825 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00018110-700a-4886-8109-b312f6f8ef02 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:41.087427 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00018110-700a-4886-8109-b312f6f8ef02 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:41.094081 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1bcb9b-1368-4695-a890-cccf02cf8682 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:41.094871 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1bcb9b-1368-4695-a890-cccf02cf8682 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:41.095858 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1bcb9b-1368-4695-a890-cccf02cf8682 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:41.096314 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1bcb9b-1368-4695-a890-cccf02cf8682 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:41.119917 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:41.163126 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1bcb9b-1368-4695-a890-cccf02cf8682 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:41.164639 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1bcb9b-1368-4695-a890-cccf02cf8682 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:41.174986 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1352/2725] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:41 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:43.416234 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc22c1-063c-4387-b45f-b9754248773c service cinder] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:43.417634 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-20bc22c1-063c-4387-b45f-b9754248773c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:43.462900 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:43.483945 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc22c1-063c-4387-b45f-b9754248773c service cinder] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:43.488337 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d052459a-4d82-4f1e-996a-399a89f0e87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:43.489015 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052459a-4d82-4f1e-996a-399a89f0e87e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:43.489297 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052459a-4d82-4f1e-996a-399a89f0e87e admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:43.489572 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052459a-4d82-4f1e-996a-399a89f0e87e admin admin] PATH_INFO: `/v3/users/8b4acd252f0d4cdba3441b74a675db98` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:43.490723 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d052459a-4d82-4f1e-996a-399a89f0e87e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b4acd252f0d4cdba3441b74a675db98)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:43.492649 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d052459a-4d82-4f1e-996a-399a89f0e87e admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:43.546013 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d052459a-4d82-4f1e-996a-399a89f0e87e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b4acd252f0d4cdba3441b74a675db98'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:43.546722 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d052459a-4d82-4f1e-996a-399a89f0e87e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b4acd252f0d4cdba3441b74a675db98'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:43.558952 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d052459a-4d82-4f1e-996a-399a89f0e87e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b4acd252f0d4cdba3441b74a675db98'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:43.566649 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d052459a-4d82-4f1e-996a-399a89f0e87e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b4acd252f0d4cdba3441b74a675db98'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:43.567283 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d052459a-4d82-4f1e-996a-399a89f0e87e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b4acd252f0d4cdba3441b74a675db98'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:43.579800 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1374/2726] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:43 2022] DELETE /identity/v3/users/8b4acd252f0d4cdba3441b74a675db98 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:43.584883 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45128169-d200-4f3b-a1f7-f5a06779ce26 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:43.585387 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45128169-d200-4f3b-a1f7-f5a06779ce26 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:43.585500 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45128169-d200-4f3b-a1f7-f5a06779ce26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:43.604690 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-45128169-d200-4f3b-a1f7-f5a06779ce26 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:43.605888 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-45128169-d200-4f3b-a1f7-f5a06779ce26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:43.663217 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1353/2727] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:43.672868 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d052459a-4d82-4f1e-996a-399a89f0e87e admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:43.674257 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-d052459a-4d82-4f1e-996a-399a89f0e87e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:43.731868 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d052459a-4d82-4f1e-996a-399a89f0e87e admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:43.735486 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396678a4-940b-4542-ba28-6b866e6375e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:43.736100 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396678a4-940b-4542-ba28-6b866e6375e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:43.736385 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396678a4-940b-4542-ba28-6b866e6375e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:43.736689 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396678a4-940b-4542-ba28-6b866e6375e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:43.745636 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396678a4-940b-4542-ba28-6b866e6375e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:43.747054 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396678a4-940b-4542-ba28-6b866e6375e3 service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:43.756969 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1375/2728] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:44.035148 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45128169-d200-4f3b-a1f7-f5a06779ce26 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:44.036825 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-45128169-d200-4f3b-a1f7-f5a06779ce26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:44.052253 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45128169-d200-4f3b-a1f7-f5a06779ce26 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:44.059303 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11341ec8-e0a0-4348-9daa-35a6942fae83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:44.059767 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11341ec8-e0a0-4348-9daa-35a6942fae83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:44.060024 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11341ec8-e0a0-4348-9daa-35a6942fae83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:44.060890 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11341ec8-e0a0-4348-9daa-35a6942fae83 admin admin] PATH_INFO: `/v3/projects/0a5c2a0f8e014c3bbbd5a6b2123ad6a3` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:44.061801 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11341ec8-e0a0-4348-9daa-35a6942fae83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a5c2a0f8e014c3bbbd5a6b2123ad6a3)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:44.063752 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11341ec8-e0a0-4348-9daa-35a6942fae83 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:44.115711 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-11341ec8-e0a0-4348-9daa-35a6942fae83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a5c2a0f8e014c3bbbd5a6b2123ad6a3', 'request_id': 'req-11341ec8-e0a0-4348-9daa-35a6942fae83'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:44.123665 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-11341ec8-e0a0-4348-9daa-35a6942fae83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a5c2a0f8e014c3bbbd5a6b2123ad6a3', 'request_id': 'req-11341ec8-e0a0-4348-9daa-35a6942fae83'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:44.130928 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-11341ec8-e0a0-4348-9daa-35a6942fae83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a5c2a0f8e014c3bbbd5a6b2123ad6a3', 'request_id': 'req-11341ec8-e0a0-4348-9daa-35a6942fae83'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:44.141276 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-11341ec8-e0a0-4348-9daa-35a6942fae83 admin admin] The token cache is being invalidate because project 0a5c2a0f8e014c3bbbd5a6b2123ad6a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:44.141576 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-11341ec8-e0a0-4348-9daa-35a6942fae83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:44.142215 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-11341ec8-e0a0-4348-9daa-35a6942fae83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:44.143715 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1354/2729] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:44 2022] DELETE /identity/v3/projects/0a5c2a0f8e014c3bbbd5a6b2123ad6a3 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:44.147664 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396678a4-940b-4542-ba28-6b866e6375e3 service neutron] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:44.157314 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396678a4-940b-4542-ba28-6b866e6375e3 service neutron] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:44.213829 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f51b8-7b43-4ce6-8ff6-46b4f694685f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:44.214472 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f51b8-7b43-4ce6-8ff6-46b4f694685f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:44.214756 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f51b8-7b43-4ce6-8ff6-46b4f694685f admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:44.215057 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f51b8-7b43-4ce6-8ff6-46b4f694685f admin admin] PATH_INFO: `/v3/users/e55df4d286ba4395831cd9492aff1f6a` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:44.216200 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f51b8-7b43-4ce6-8ff6-46b4f694685f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e55df4d286ba4395831cd9492aff1f6a)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:44.218161 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f51b8-7b43-4ce6-8ff6-46b4f694685f admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:44.222740 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:44.267007 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-213f51b8-7b43-4ce6-8ff6-46b4f694685f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e55df4d286ba4395831cd9492aff1f6a'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:44.267655 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-213f51b8-7b43-4ce6-8ff6-46b4f694685f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e55df4d286ba4395831cd9492aff1f6a'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:44.279317 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-213f51b8-7b43-4ce6-8ff6-46b4f694685f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e55df4d286ba4395831cd9492aff1f6a'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:44.286659 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-213f51b8-7b43-4ce6-8ff6-46b4f694685f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e55df4d286ba4395831cd9492aff1f6a'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:44.287391 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-213f51b8-7b43-4ce6-8ff6-46b4f694685f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e55df4d286ba4395831cd9492aff1f6a'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:44.300005 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1376/2730] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:44 2022] DELETE /identity/v3/users/e55df4d286ba4395831cd9492aff1f6a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:44.564494 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11341ec8-e0a0-4348-9daa-35a6942fae83 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:44.566089 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-11341ec8-e0a0-4348-9daa-35a6942fae83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:44.580969 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11341ec8-e0a0-4348-9daa-35a6942fae83 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:44.624384 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:44.640272 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75d260-16e2-4f17-b37e-7ff4ee47748f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:44.640955 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75d260-16e2-4f17-b37e-7ff4ee47748f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:44.641239 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75d260-16e2-4f17-b37e-7ff4ee47748f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:44.641516 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75d260-16e2-4f17-b37e-7ff4ee47748f admin admin] PATH_INFO: `/v3/projects/efefa095b5a14057aaadc08cc7fc9787` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:44.642664 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac75d260-16e2-4f17-b37e-7ff4ee47748f admin admin] RBAC: Authorizing `identity:delete_project(project_id=efefa095b5a14057aaadc08cc7fc9787)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:44.644615 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac75d260-16e2-4f17-b37e-7ff4ee47748f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:44.689823 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ac75d260-16e2-4f17-b37e-7ff4ee47748f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efefa095b5a14057aaadc08cc7fc9787', 'request_id': 'req-ac75d260-16e2-4f17-b37e-7ff4ee47748f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:44.696728 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ac75d260-16e2-4f17-b37e-7ff4ee47748f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efefa095b5a14057aaadc08cc7fc9787', 'request_id': 'req-ac75d260-16e2-4f17-b37e-7ff4ee47748f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:44.702996 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ac75d260-16e2-4f17-b37e-7ff4ee47748f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efefa095b5a14057aaadc08cc7fc9787', 'request_id': 'req-ac75d260-16e2-4f17-b37e-7ff4ee47748f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:44.713435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ac75d260-16e2-4f17-b37e-7ff4ee47748f admin admin] The token cache is being invalidate because project efefa095b5a14057aaadc08cc7fc9787 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:44.713729 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ac75d260-16e2-4f17-b37e-7ff4ee47748f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:44.714390 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ac75d260-16e2-4f17-b37e-7ff4ee47748f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:44.715882 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1355/2731] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:44 2022] DELETE /identity/v3/projects/efefa095b5a14057aaadc08cc7fc9787 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:44.720738 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3017f42-315d-4ec8-bf31-5601507ccce5 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:44.721209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3017f42-315d-4ec8-bf31-5601507ccce5 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:44.721311 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3017f42-315d-4ec8-bf31-5601507ccce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:44.741307 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-e3017f42-315d-4ec8-bf31-5601507ccce5 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:44.742510 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-e3017f42-315d-4ec8-bf31-5601507ccce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:44.808171 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1377/2732] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:44.815757 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9072e-17a3-4151-ad3b-c88402d710a1 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:44.816044 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9072e-17a3-4151-ad3b-c88402d710a1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:44.816324 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9072e-17a3-4151-ad3b-c88402d710a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:44.836976 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-7db9072e-17a3-4151-ad3b-c88402d710a1 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:44.838161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7db9072e-17a3-4151-ad3b-c88402d710a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:44.846281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1356/2733] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:44.851514 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3017f42-315d-4ec8-bf31-5601507ccce5 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:44.861185 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3017f42-315d-4ec8-bf31-5601507ccce5 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:44.865770 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b18565-93c8-46a4-83cf-ca4c17954178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:44.866377 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b18565-93c8-46a4-83cf-ca4c17954178 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:44.866665 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b18565-93c8-46a4-83cf-ca4c17954178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:44.866945 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b18565-93c8-46a4-83cf-ca4c17954178 admin admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:44.867645 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b18565-93c8-46a4-83cf-ca4c17954178 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:21:44.867953 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b18565-93c8-46a4-83cf-ca4c17954178 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:44.869574 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b18565-93c8-46a4-83cf-ca4c17954178 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:44.880920 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1378/2734] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:21:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:21:44.885119 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db9072e-17a3-4151-ad3b-c88402d710a1 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:44.894284 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db9072e-17a3-4151-ad3b-c88402d710a1 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:44.899354 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f36ff-8738-4a72-b563-b93e18992705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:44.899950 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f36ff-8738-4a72-b563-b93e18992705 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:44.900256 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f36ff-8738-4a72-b563-b93e18992705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:44.900563 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f36ff-8738-4a72-b563-b93e18992705 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:44.901231 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f36ff-8738-4a72-b563-b93e18992705 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:44.903119 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f36ff-8738-4a72-b563-b93e18992705 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:44.929366 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1357/2735] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:44 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:44.933624 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b18565-93c8-46a4-83cf-ca4c17954178 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:44.944123 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b18565-93c8-46a4-83cf-ca4c17954178 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:44.999933 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6ea5e-e8f9-4170-aa19-f64858724689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:45.000588 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6ea5e-e8f9-4170-aa19-f64858724689 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:45.000878 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6ea5e-e8f9-4170-aa19-f64858724689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:45.001164 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6ea5e-e8f9-4170-aa19-f64858724689 admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:45.001838 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c6ea5e-e8f9-4170-aa19-f64858724689 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:45.003772 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c6ea5e-e8f9-4170-aa19-f64858724689 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:45.020183 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:45.029104 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1379/2736] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:44 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:45.030242 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f36ff-8738-4a72-b563-b93e18992705 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:45.041291 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f36ff-8738-4a72-b563-b93e18992705 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:45.045944 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02633b8c-b926-4c18-b794-7226a26fdb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:45.046552 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02633b8c-b926-4c18-b794-7226a26fdb8e admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:45.046841 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02633b8c-b926-4c18-b794-7226a26fdb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:45.047122 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02633b8c-b926-4c18-b794-7226a26fdb8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:45.047820 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02633b8c-b926-4c18-b794-7226a26fdb8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:45.049445 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02633b8c-b926-4c18-b794-7226a26fdb8e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:45.060086 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1358/2737] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:45.063914 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6ea5e-e8f9-4170-aa19-f64858724689 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:45.073872 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6ea5e-e8f9-4170-aa19-f64858724689 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:45.078396 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff74b85-1eac-44d4-89c9-b9773674ad36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:45.079004 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff74b85-1eac-44d4-89c9-b9773674ad36 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:45.079283 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff74b85-1eac-44d4-89c9-b9773674ad36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:45.079560 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff74b85-1eac-44d4-89c9-b9773674ad36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:45.080250 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff74b85-1eac-44d4-89c9-b9773674ad36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:45.081862 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff74b85-1eac-44d4-89c9-b9773674ad36 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:45.093133 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1380/2738] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:45.097274 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02633b8c-b926-4c18-b794-7226a26fdb8e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:45.107710 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02633b8c-b926-4c18-b794-7226a26fdb8e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:45.112313 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f0a5f-ee30-4a7a-8ddf-6ba869038a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:45.113001 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f0a5f-ee30-4a7a-8ddf-6ba869038a0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:45.113314 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f0a5f-ee30-4a7a-8ddf-6ba869038a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:45.113594 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f0a5f-ee30-4a7a-8ddf-6ba869038a0f admin admin] PATH_INFO: `/v3/projects/de5aeb731d2c409189934d5845a1fcd8/users/b15e34f618764704bc876d10216cbf82/roles/926b9f36c24847c582c84af904250abc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:45.127595 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f0a5f-ee30-4a7a-8ddf-6ba869038a0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=de5aeb731d2c409189934d5845a1fcd8, user_id=b15e34f618764704bc876d10216cbf82, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:45.130823 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f0a5f-ee30-4a7a-8ddf-6ba869038a0f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:45.144298 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1359/2739] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:45 2022] PUT /identity/v3/projects/de5aeb731d2c409189934d5845a1fcd8/users/b15e34f618764704bc876d10216cbf82/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:45.149957 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79ca7ec-46f8-443f-989d-b4b4ae74a120 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:45.150245 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79ca7ec-46f8-443f-989d-b4b4ae74a120 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:45.150521 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79ca7ec-46f8-443f-989d-b4b4ae74a120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:45.168949 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-b79ca7ec-46f8-443f-989d-b4b4ae74a120 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:45.170167 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-b79ca7ec-46f8-443f-989d-b4b4ae74a120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:45.232838 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1381/2740] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:45.242454 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f0a5f-ee30-4a7a-8ddf-6ba869038a0f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:45.244274 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b90f0a5f-ee30-4a7a-8ddf-6ba869038a0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:45.301150 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f0a5f-ee30-4a7a-8ddf-6ba869038a0f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:45.304806 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f08b7-a051-4537-ae73-0f9765edaa80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:45.305418 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f08b7-a051-4537-ae73-0f9765edaa80 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:45.305703 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f08b7-a051-4537-ae73-0f9765edaa80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:45.305983 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f08b7-a051-4537-ae73-0f9765edaa80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:45.315172 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f08b7-a051-4537-ae73-0f9765edaa80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:45.317060 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f08b7-a051-4537-ae73-0f9765edaa80 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:45.326428 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1360/2741] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:45.949304 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304a0aa-14ee-4ffc-b746-5c4334a48ef7 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:45.949825 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304a0aa-14ee-4ffc-b746-5c4334a48ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:45.949952 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304a0aa-14ee-4ffc-b746-5c4334a48ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:45.978760 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-0304a0aa-14ee-4ffc-b746-5c4334a48ef7 None None] MFA Rules not processed for user `b15e34f618764704bc876d10216cbf82`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:45.979980 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-0304a0aa-14ee-4ffc-b746-5c4334a48ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:46.049689 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1382/2742] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:46.054910 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f08b7-a051-4537-ae73-0f9765edaa80 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:46.065325 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f08b7-a051-4537-ae73-0f9765edaa80 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:46.070384 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb01925-9b2e-45c2-99ce-61e0ccf2a6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:46.071009 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb01925-9b2e-45c2-99ce-61e0ccf2a6fe admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:46.071303 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb01925-9b2e-45c2-99ce-61e0ccf2a6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:46.071547 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb01925-9b2e-45c2-99ce-61e0ccf2a6fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:46.072224 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb01925-9b2e-45c2-99ce-61e0ccf2a6fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:46.074128 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb01925-9b2e-45c2-99ce-61e0ccf2a6fe admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:46.086840 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:46.107627 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1361/2743] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:46 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:46.112257 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304a0aa-14ee-4ffc-b746-5c4334a48ef7 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:46.127246 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304a0aa-14ee-4ffc-b746-5c4334a48ef7 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:46.185735 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75267fd-c537-43f9-90d2-8627d6dcb68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:46.186360 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75267fd-c537-43f9-90d2-8627d6dcb68c admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:46.186647 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75267fd-c537-43f9-90d2-8627d6dcb68c admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:46.186927 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75267fd-c537-43f9-90d2-8627d6dcb68c admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:46.187606 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75267fd-c537-43f9-90d2-8627d6dcb68c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:46.189555 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75267fd-c537-43f9-90d2-8627d6dcb68c admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:46.213547 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1383/2744] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:46 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:21:46.214722 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb01925-9b2e-45c2-99ce-61e0ccf2a6fe admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:46.225547 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb01925-9b2e-45c2-99ce-61e0ccf2a6fe admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:46.234314 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a87114-701d-4fbc-a224-ccfd21733630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:46.234933 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a87114-701d-4fbc-a224-ccfd21733630 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:46.235215 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a87114-701d-4fbc-a224-ccfd21733630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:46.235491 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a87114-701d-4fbc-a224-ccfd21733630 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:46.236189 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a87114-701d-4fbc-a224-ccfd21733630 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:46.237830 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a87114-701d-4fbc-a224-ccfd21733630 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:46.248859 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1362/2745] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:46.253005 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75267fd-c537-43f9-90d2-8627d6dcb68c admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:46.263091 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75267fd-c537-43f9-90d2-8627d6dcb68c admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:46.267768 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7bd90-02db-4758-a232-bfea5adbfb62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:46.268439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7bd90-02db-4758-a232-bfea5adbfb62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:46.268783 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7bd90-02db-4758-a232-bfea5adbfb62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:46.269060 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7bd90-02db-4758-a232-bfea5adbfb62 admin admin] PATH_INFO: `/v3/projects/9df74a9b9fe14af0b9374b365f94c8bc/users/77e055fa535545ef924a95b611dfb713/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:46.272744 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:46.286437 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7bd90-02db-4758-a232-bfea5adbfb62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9df74a9b9fe14af0b9374b365f94c8bc, user_id=77e055fa535545ef924a95b611dfb713, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:46.289200 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7bd90-02db-4758-a232-bfea5adbfb62 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:46.301855 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1384/2746] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:46 2022] PUT /identity/v3/projects/9df74a9b9fe14af0b9374b365f94c8bc/users/77e055fa535545ef924a95b611dfb713/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:46.305798 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a87114-701d-4fbc-a224-ccfd21733630 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:46.316199 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a87114-701d-4fbc-a224-ccfd21733630 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:46.371644 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51dbe0-733e-49a2-8fb6-cf2d5aade7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:46.372221 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51dbe0-733e-49a2-8fb6-cf2d5aade7f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:46.372535 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51dbe0-733e-49a2-8fb6-cf2d5aade7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:46.372818 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51dbe0-733e-49a2-8fb6-cf2d5aade7f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:46.373519 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec51dbe0-733e-49a2-8fb6-cf2d5aade7f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:46.375124 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec51dbe0-733e-49a2-8fb6-cf2d5aade7f7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:46.385244 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1363/2747] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:46.389475 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7bd90-02db-4758-a232-bfea5adbfb62 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:46.400012 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7bd90-02db-4758-a232-bfea5adbfb62 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:46.404722 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25827e-8ed4-4216-9dff-0a5319ee9bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:46.405388 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25827e-8ed4-4216-9dff-0a5319ee9bb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:46.405669 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25827e-8ed4-4216-9dff-0a5319ee9bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:46.405947 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25827e-8ed4-4216-9dff-0a5319ee9bb1 admin admin] PATH_INFO: `/v3/projects/9df74a9b9fe14af0b9374b365f94c8bc/users/77e055fa535545ef924a95b611dfb713/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:46.407899 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c25827e-8ed4-4216-9dff-0a5319ee9bb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9df74a9b9fe14af0b9374b365f94c8bc, user_id=77e055fa535545ef924a95b611dfb713, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:46.410574 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c25827e-8ed4-4216-9dff-0a5319ee9bb1 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:46.421019 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1385/2748] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:46 2022] PUT /identity/v3/projects/9df74a9b9fe14af0b9374b365f94c8bc/users/77e055fa535545ef924a95b611dfb713/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:21:47.049268 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80b1d9-619b-40fc-9f81-78a3865dc901 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:47.049565 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80b1d9-619b-40fc-9f81-78a3865dc901 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:47.049841 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80b1d9-619b-40fc-9f81-78a3865dc901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:47.079343 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-2b80b1d9-619b-40fc-9f81-78a3865dc901 None None] MFA Rules not processed for user `77e055fa535545ef924a95b611dfb713`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:47.080734 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2b80b1d9-619b-40fc-9f81-78a3865dc901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:47.164889 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1364/2749] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:47 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:47.177493 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25827e-8ed4-4216-9dff-0a5319ee9bb1 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:47.179641 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-8c25827e-8ed4-4216-9dff-0a5319ee9bb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:47.248409 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25827e-8ed4-4216-9dff-0a5319ee9bb1 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:47.252416 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91be97f3-ff45-4f07-9b0c-429cfdf12e85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:47.253134 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be97f3-ff45-4f07-9b0c-429cfdf12e85 service cinder] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:47.253589 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be97f3-ff45-4f07-9b0c-429cfdf12e85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:47.254533 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be97f3-ff45-4f07-9b0c-429cfdf12e85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:47.321801 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91be97f3-ff45-4f07-9b0c-429cfdf12e85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:47.323237 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91be97f3-ff45-4f07-9b0c-429cfdf12e85 service cinder] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:47.332351 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1386/2750] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:47.401858 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b80b1d9-619b-40fc-9f81-78a3865dc901 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:47.403423 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2b80b1d9-619b-40fc-9f81-78a3865dc901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:47.473520 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b80b1d9-619b-40fc-9f81-78a3865dc901 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:47.478804 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd10a541-106f-43f7-9d39-7bc267598297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:47.479558 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd10a541-106f-43f7-9d39-7bc267598297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:47.479773 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd10a541-106f-43f7-9d39-7bc267598297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:47.480283 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd10a541-106f-43f7-9d39-7bc267598297 admin admin] PATH_INFO: `/v3/users/820e52f08ec84493b68726505679752f` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:47.481811 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd10a541-106f-43f7-9d39-7bc267598297 admin admin] RBAC: Authorizing `identity:delete_user(user_id=820e52f08ec84493b68726505679752f)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:47.483915 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd10a541-106f-43f7-9d39-7bc267598297 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:47.486795 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:47.537042 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dd10a541-106f-43f7-9d39-7bc267598297 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '820e52f08ec84493b68726505679752f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:47.544810 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dd10a541-106f-43f7-9d39-7bc267598297 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '820e52f08ec84493b68726505679752f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:47.545592 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dd10a541-106f-43f7-9d39-7bc267598297 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '820e52f08ec84493b68726505679752f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:47.553286 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dd10a541-106f-43f7-9d39-7bc267598297 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '820e52f08ec84493b68726505679752f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:47.553912 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dd10a541-106f-43f7-9d39-7bc267598297 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '820e52f08ec84493b68726505679752f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:47.587882 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1365/2751] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:47 2022] DELETE /identity/v3/users/820e52f08ec84493b68726505679752f => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:47.592329 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc204a-662b-4a1c-9fdf-7b63488c8050 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:47.592660 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc204a-662b-4a1c-9fdf-7b63488c8050 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:47.593578 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc204a-662b-4a1c-9fdf-7b63488c8050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:47.617700 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-f6bc204a-662b-4a1c-9fdf-7b63488c8050 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:47.619333 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-f6bc204a-662b-4a1c-9fdf-7b63488c8050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:47.696787 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1387/2752] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:47.706549 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd10a541-106f-43f7-9d39-7bc267598297 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:47.708814 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-dd10a541-106f-43f7-9d39-7bc267598297 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:47.770867 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd10a541-106f-43f7-9d39-7bc267598297 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:47.774739 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470b772-7d61-405f-a55c-1ffbe8b6b822 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:47.775366 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470b772-7d61-405f-a55c-1ffbe8b6b822 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:47.775653 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470b772-7d61-405f-a55c-1ffbe8b6b822 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:47.775931 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470b772-7d61-405f-a55c-1ffbe8b6b822 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:47.785905 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3470b772-7d61-405f-a55c-1ffbe8b6b822 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:47.787492 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3470b772-7d61-405f-a55c-1ffbe8b6b822 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:47.802262 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1366/2753] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:21:48.081694 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc204a-662b-4a1c-9fdf-7b63488c8050 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:48.083631 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-f6bc204a-662b-4a1c-9fdf-7b63488c8050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:48.099345 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc204a-662b-4a1c-9fdf-7b63488c8050 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:48.103992 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3025a197-2864-44fd-b24f-942b153fd1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:48.104670 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3025a197-2864-44fd-b24f-942b153fd1d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:48.104969 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3025a197-2864-44fd-b24f-942b153fd1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:48.105313 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3025a197-2864-44fd-b24f-942b153fd1d8 admin admin] PATH_INFO: `/v3/projects/95b5987a051942b196b5d956cc2fd5ce` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:48.106486 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3025a197-2864-44fd-b24f-942b153fd1d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95b5987a051942b196b5d956cc2fd5ce)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:48.108518 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3025a197-2864-44fd-b24f-942b153fd1d8 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:48.135729 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:48.161373 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-3025a197-2864-44fd-b24f-942b153fd1d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95b5987a051942b196b5d956cc2fd5ce', 'request_id': 'req-3025a197-2864-44fd-b24f-942b153fd1d8'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:48.169003 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-3025a197-2864-44fd-b24f-942b153fd1d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95b5987a051942b196b5d956cc2fd5ce', 'request_id': 'req-3025a197-2864-44fd-b24f-942b153fd1d8'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:48.174588 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-3025a197-2864-44fd-b24f-942b153fd1d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95b5987a051942b196b5d956cc2fd5ce', 'request_id': 'req-3025a197-2864-44fd-b24f-942b153fd1d8'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:48.193898 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-3025a197-2864-44fd-b24f-942b153fd1d8 admin admin] The token cache is being invalidate because project 95b5987a051942b196b5d956cc2fd5ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:48.194578 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-3025a197-2864-44fd-b24f-942b153fd1d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:48.194803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-3025a197-2864-44fd-b24f-942b153fd1d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:48.196381 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1388/2754] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:48 2022] DELETE /identity/v3/projects/95b5987a051942b196b5d956cc2fd5ce => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:48.200694 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470b772-7d61-405f-a55c-1ffbe8b6b822 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:48.211156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470b772-7d61-405f-a55c-1ffbe8b6b822 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:48.266604 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e6c52-8a89-419c-839b-b6dd93fc143b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:48.267149 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e6c52-8a89-419c-839b-b6dd93fc143b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:48.267430 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e6c52-8a89-419c-839b-b6dd93fc143b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:48.267707 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e6c52-8a89-419c-839b-b6dd93fc143b admin admin] PATH_INFO: `/v3/users/d6b22b0be97947d1b342cd25e65e32ab` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:48.268897 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e6c52-8a89-419c-839b-b6dd93fc143b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6b22b0be97947d1b342cd25e65e32ab)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:48.271374 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e6c52-8a89-419c-839b-b6dd93fc143b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:48.299193 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:48.318702 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-274e6c52-8a89-419c-839b-b6dd93fc143b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6b22b0be97947d1b342cd25e65e32ab'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:48.325812 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-274e6c52-8a89-419c-839b-b6dd93fc143b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6b22b0be97947d1b342cd25e65e32ab'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:48.326478 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-274e6c52-8a89-419c-839b-b6dd93fc143b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6b22b0be97947d1b342cd25e65e32ab'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:48.334136 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-274e6c52-8a89-419c-839b-b6dd93fc143b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6b22b0be97947d1b342cd25e65e32ab'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:48.334867 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-274e6c52-8a89-419c-839b-b6dd93fc143b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6b22b0be97947d1b342cd25e65e32ab'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:48.352177 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1367/2755] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:48 2022] DELETE /identity/v3/users/d6b22b0be97947d1b342cd25e65e32ab => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:48.632285 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3025a197-2864-44fd-b24f-942b153fd1d8 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:48.633856 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-3025a197-2864-44fd-b24f-942b153fd1d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:48.648314 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3025a197-2864-44fd-b24f-942b153fd1d8 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:48.716766 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e6c52-8a89-419c-839b-b6dd93fc143b admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:48.718608 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-274e6c52-8a89-419c-839b-b6dd93fc143b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:48.723540 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7cbb4-0a4e-458a-bcf4-1133e45f3fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:48.724443 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7cbb4-0a4e-458a-bcf4-1133e45f3fbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:48.724754 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7cbb4-0a4e-458a-bcf4-1133e45f3fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:48.725559 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7cbb4-0a4e-458a-bcf4-1133e45f3fbd admin admin] PATH_INFO: `/v3/projects/3554a8e300da45939f332d10897002ce` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:48.726732 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7cbb4-0a4e-458a-bcf4-1133e45f3fbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3554a8e300da45939f332d10897002ce)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:48.728647 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7cbb4-0a4e-458a-bcf4-1133e45f3fbd admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:48.774535 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-eca7cbb4-0a4e-458a-bcf4-1133e45f3fbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3554a8e300da45939f332d10897002ce', 'request_id': 'req-eca7cbb4-0a4e-458a-bcf4-1133e45f3fbd'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:48.778446 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e6c52-8a89-419c-839b-b6dd93fc143b admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:48.780934 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-eca7cbb4-0a4e-458a-bcf4-1133e45f3fbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3554a8e300da45939f332d10897002ce', 'request_id': 'req-eca7cbb4-0a4e-458a-bcf4-1133e45f3fbd'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:48.786806 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-eca7cbb4-0a4e-458a-bcf4-1133e45f3fbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3554a8e300da45939f332d10897002ce', 'request_id': 'req-eca7cbb4-0a4e-458a-bcf4-1133e45f3fbd'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:48.800995 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-eca7cbb4-0a4e-458a-bcf4-1133e45f3fbd admin admin] The token cache is being invalidate because project 3554a8e300da45939f332d10897002ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:48.801397 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-eca7cbb4-0a4e-458a-bcf4-1133e45f3fbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:48.802045 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-eca7cbb4-0a4e-458a-bcf4-1133e45f3fbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:48.806628 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1389/2756] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:48 2022] DELETE /identity/v3/projects/3554a8e300da45939f332d10897002ce => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:48.813880 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe72eb0-3234-4c2c-b00a-0aabc3844e59 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:48.814353 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe72eb0-3234-4c2c-b00a-0aabc3844e59 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:48.814703 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe72eb0-3234-4c2c-b00a-0aabc3844e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:48.834808 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-9fe72eb0-3234-4c2c-b00a-0aabc3844e59 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:48.836051 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-9fe72eb0-3234-4c2c-b00a-0aabc3844e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:48.845347 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628905f4-ffe5-4f55-aa65-2bd6e5349e63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:48.846107 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628905f4-ffe5-4f55-aa65-2bd6e5349e63 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:48.846518 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628905f4-ffe5-4f55-aa65-2bd6e5349e63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:48.846804 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628905f4-ffe5-4f55-aa65-2bd6e5349e63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:48.849181 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-628905f4-ffe5-4f55-aa65-2bd6e5349e63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:48.876200 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:48.908770 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1390/2757] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:21:48.917737 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad658f52-704d-47d0-9669-f0f4a8970ede None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:48.918063 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad658f52-704d-47d0-9669-f0f4a8970ede None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:48.918349 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad658f52-704d-47d0-9669-f0f4a8970ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:48.921000 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628905f4-ffe5-4f55-aa65-2bd6e5349e63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:48.922476 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628905f4-ffe5-4f55-aa65-2bd6e5349e63 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:48.933119 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1368/2758] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:48 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:48.938805 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-ad658f52-704d-47d0-9669-f0f4a8970ede None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:48.940140 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-ad658f52-704d-47d0-9669-f0f4a8970ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:48.948129 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1391/2759] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:21:48.952663 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628905f4-ffe5-4f55-aa65-2bd6e5349e63 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:48.967186 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628905f4-ffe5-4f55-aa65-2bd6e5349e63 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:48.972124 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def0bd8e-9a78-43e5-8a1b-62e45c2cca58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:48.973280 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0bd8e-9a78-43e5-8a1b-62e45c2cca58 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:48.973570 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0bd8e-9a78-43e5-8a1b-62e45c2cca58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:48.973851 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0bd8e-9a78-43e5-8a1b-62e45c2cca58 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:48.974644 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def0bd8e-9a78-43e5-8a1b-62e45c2cca58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:21:48.975030 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def0bd8e-9a78-43e5-8a1b-62e45c2cca58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:48.976903 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def0bd8e-9a78-43e5-8a1b-62e45c2cca58 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:48.979931 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:48.998893 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1369/2760] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:21:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:21:48.999960 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad658f52-704d-47d0-9669-f0f4a8970ede None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:49.010176 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad658f52-704d-47d0-9669-f0f4a8970ede None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:49.014963 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0944aa18-615d-4e47-bb30-b60f5094c083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:49.015585 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944aa18-615d-4e47-bb30-b60f5094c083 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:49.015870 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944aa18-615d-4e47-bb30-b60f5094c083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:49.016151 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944aa18-615d-4e47-bb30-b60f5094c083 admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:49.016870 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0944aa18-615d-4e47-bb30-b60f5094c083 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:49.018789 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0944aa18-615d-4e47-bb30-b60f5094c083 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:49.048675 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1392/2761] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:48 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:49.052644 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def0bd8e-9a78-43e5-8a1b-62e45c2cca58 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:49.062869 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def0bd8e-9a78-43e5-8a1b-62e45c2cca58 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:49.122674 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e1f95-d3f6-4082-865e-c0b4aacb3093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:49.123294 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e1f95-d3f6-4082-865e-c0b4aacb3093 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:49.123578 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e1f95-d3f6-4082-865e-c0b4aacb3093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:49.123853 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e1f95-d3f6-4082-865e-c0b4aacb3093 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:49.124611 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e1f95-d3f6-4082-865e-c0b4aacb3093 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:49.126577 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e1f95-d3f6-4082-865e-c0b4aacb3093 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:49.147649 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1370/2762] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:49 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:49.152079 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0944aa18-615d-4e47-bb30-b60f5094c083 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:49.163114 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0944aa18-615d-4e47-bb30-b60f5094c083 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:49.167803 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef1463-bd4a-4c77-abdf-599b1d53158c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:49.168398 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef1463-bd4a-4c77-abdf-599b1d53158c admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:49.168735 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef1463-bd4a-4c77-abdf-599b1d53158c admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:49.169020 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef1463-bd4a-4c77-abdf-599b1d53158c admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:49.169722 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ef1463-bd4a-4c77-abdf-599b1d53158c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:49.171301 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ef1463-bd4a-4c77-abdf-599b1d53158c admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:49.182128 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1393/2763] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:21:49.186326 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e1f95-d3f6-4082-865e-c0b4aacb3093 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:49.196253 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e1f95-d3f6-4082-865e-c0b4aacb3093 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:49.201044 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e95bc-87a0-4d9c-8a17-51825f80640f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:49.201755 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e95bc-87a0-4d9c-8a17-51825f80640f admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:49.202118 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e95bc-87a0-4d9c-8a17-51825f80640f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:49.202437 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e95bc-87a0-4d9c-8a17-51825f80640f admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:49.203138 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e95bc-87a0-4d9c-8a17-51825f80640f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:49.204761 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e95bc-87a0-4d9c-8a17-51825f80640f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:49.215535 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1371/2764] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:49.219591 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef1463-bd4a-4c77-abdf-599b1d53158c admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:49.230096 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef1463-bd4a-4c77-abdf-599b1d53158c admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:49.234628 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90af2e2-93fb-4cbb-aa60-06b60b36edf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:49.235285 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90af2e2-93fb-4cbb-aa60-06b60b36edf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:49.235573 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90af2e2-93fb-4cbb-aa60-06b60b36edf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:49.235856 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90af2e2-93fb-4cbb-aa60-06b60b36edf7 admin admin] PATH_INFO: `/v3/projects/8d0badb911c94fc985cc8b3e40a1f085/users/8347dd1db0cd4f02b784336c9c0cfebe/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:49.250734 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90af2e2-93fb-4cbb-aa60-06b60b36edf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d0badb911c94fc985cc8b3e40a1f085, user_id=8347dd1db0cd4f02b784336c9c0cfebe, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:49.253414 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90af2e2-93fb-4cbb-aa60-06b60b36edf7 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:49.265810 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1394/2765] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:49 2022] PUT /identity/v3/projects/8d0badb911c94fc985cc8b3e40a1f085/users/8347dd1db0cd4f02b784336c9c0cfebe/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:49.277516 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a373b-bb6c-471d-a133-7016dc1996ef None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:49.277806 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a373b-bb6c-471d-a133-7016dc1996ef None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:49.278084 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a373b-bb6c-471d-a133-7016dc1996ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:49.305452 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-940a373b-bb6c-471d-a133-7016dc1996ef None None] MFA Rules not processed for user `8347dd1db0cd4f02b784336c9c0cfebe`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:49.306602 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-940a373b-bb6c-471d-a133-7016dc1996ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:49.369023 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1372/2766] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:49.373457 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90af2e2-93fb-4cbb-aa60-06b60b36edf7 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:49.434499 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90af2e2-93fb-4cbb-aa60-06b60b36edf7 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:49.439075 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34cfacf-6640-40c8-bfce-2d5f8ad371c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:49.439683 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34cfacf-6640-40c8-bfce-2d5f8ad371c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:49.439967 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34cfacf-6640-40c8-bfce-2d5f8ad371c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:49.440244 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34cfacf-6640-40c8-bfce-2d5f8ad371c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:49.440940 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34cfacf-6640-40c8-bfce-2d5f8ad371c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:49.442853 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34cfacf-6640-40c8-bfce-2d5f8ad371c1 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:49.472301 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1395/2767] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:21:49 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:21:49.477208 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a373b-bb6c-471d-a133-7016dc1996ef None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:49.488430 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a373b-bb6c-471d-a133-7016dc1996ef None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:49.544778 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2368b688-43d9-4558-ade3-7dbbdb17742c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:49.545405 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2368b688-43d9-4558-ade3-7dbbdb17742c admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:49.545692 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2368b688-43d9-4558-ade3-7dbbdb17742c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:49.545971 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2368b688-43d9-4558-ade3-7dbbdb17742c admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:49.546634 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2368b688-43d9-4558-ade3-7dbbdb17742c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:49.548603 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2368b688-43d9-4558-ade3-7dbbdb17742c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:49.558828 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:49.570024 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1373/2768] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:21:49 2022] POST /identity/v3/users => generated 298 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:21:49.574192 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34cfacf-6640-40c8-bfce-2d5f8ad371c1 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:49.585007 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34cfacf-6640-40c8-bfce-2d5f8ad371c1 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:49.589527 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cdec41-221f-4075-9d6c-6e436f9ac08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:49.590128 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdec41-221f-4075-9d6c-6e436f9ac08a admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:49.590410 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdec41-221f-4075-9d6c-6e436f9ac08a admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:49.590685 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdec41-221f-4075-9d6c-6e436f9ac08a admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:49.591376 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cdec41-221f-4075-9d6c-6e436f9ac08a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:49.592993 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cdec41-221f-4075-9d6c-6e436f9ac08a admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:49.604375 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1396/2769] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:49.608410 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2368b688-43d9-4558-ade3-7dbbdb17742c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:49.618304 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2368b688-43d9-4558-ade3-7dbbdb17742c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:49.623983 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e4a01-eeea-46b7-83e0-5cff042a3d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:49.624668 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e4a01-eeea-46b7-83e0-5cff042a3d0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:49.624955 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e4a01-eeea-46b7-83e0-5cff042a3d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:49.625234 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e4a01-eeea-46b7-83e0-5cff042a3d0a admin admin] PATH_INFO: `/v3/projects/f785cc0b629d42c6b457d0213beb23b9/users/75ca572861bc460ca23a18cf2809e2a5/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:49.640526 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668e4a01-eeea-46b7-83e0-5cff042a3d0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f785cc0b629d42c6b457d0213beb23b9, user_id=75ca572861bc460ca23a18cf2809e2a5, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:49.643159 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668e4a01-eeea-46b7-83e0-5cff042a3d0a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:49.656200 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1374/2770] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:49 2022] PUT /identity/v3/projects/f785cc0b629d42c6b457d0213beb23b9/users/75ca572861bc460ca23a18cf2809e2a5/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:49.660118 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cdec41-221f-4075-9d6c-6e436f9ac08a admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:49.670010 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cdec41-221f-4075-9d6c-6e436f9ac08a admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:49.716191 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:49.727594 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe46c4e-955b-441f-9da6-c22b9a641c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:49.728221 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe46c4e-955b-441f-9da6-c22b9a641c94 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:49.728530 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe46c4e-955b-441f-9da6-c22b9a641c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:49.728816 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe46c4e-955b-441f-9da6-c22b9a641c94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:49.729519 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe46c4e-955b-441f-9da6-c22b9a641c94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:49.731124 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe46c4e-955b-441f-9da6-c22b9a641c94 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:49.740843 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1397/2771] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:21:49.744958 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e4a01-eeea-46b7-83e0-5cff042a3d0a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:49.755538 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e4a01-eeea-46b7-83e0-5cff042a3d0a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:49.760016 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9458fe16-97ff-4fe5-b731-7406d9259da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:49.760705 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458fe16-97ff-4fe5-b731-7406d9259da5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:49.760987 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458fe16-97ff-4fe5-b731-7406d9259da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:49.761263 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458fe16-97ff-4fe5-b731-7406d9259da5 admin admin] PATH_INFO: `/v3/projects/f785cc0b629d42c6b457d0213beb23b9/users/75ca572861bc460ca23a18cf2809e2a5/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:49.763166 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9458fe16-97ff-4fe5-b731-7406d9259da5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f785cc0b629d42c6b457d0213beb23b9, user_id=75ca572861bc460ca23a18cf2809e2a5, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:49.765814 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9458fe16-97ff-4fe5-b731-7406d9259da5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:49.776333 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1375/2772] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:21:49 2022] PUT /identity/v3/projects/f785cc0b629d42c6b457d0213beb23b9/users/75ca572861bc460ca23a18cf2809e2a5/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:49.786513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e39060-f17f-40d1-9877-5f79d75cd6dd None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:49.786808 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e39060-f17f-40d1-9877-5f79d75cd6dd None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:49.787124 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e39060-f17f-40d1-9877-5f79d75cd6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:49.822300 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-f7e39060-f17f-40d1-9877-5f79d75cd6dd None None] MFA Rules not processed for user `75ca572861bc460ca23a18cf2809e2a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:49.823531 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-f7e39060-f17f-40d1-9877-5f79d75cd6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:49.895834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1398/2773] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:49 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:49.906346 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9458fe16-97ff-4fe5-b731-7406d9259da5 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:49.908063 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9458fe16-97ff-4fe5-b731-7406d9259da5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:49.966581 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9458fe16-97ff-4fe5-b731-7406d9259da5 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:49.970266 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39703ae2-a9ad-435f-ad43-82c6b8515acc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:49.970914 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39703ae2-a9ad-435f-ad43-82c6b8515acc service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:49.971196 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39703ae2-a9ad-435f-ad43-82c6b8515acc service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:49.971474 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39703ae2-a9ad-435f-ad43-82c6b8515acc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:49.980403 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39703ae2-a9ad-435f-ad43-82c6b8515acc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:49.981854 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39703ae2-a9ad-435f-ad43-82c6b8515acc service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:49.991673 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1376/2774] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:49 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:50.091732 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e39060-f17f-40d1-9877-5f79d75cd6dd None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:50.101371 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e39060-f17f-40d1-9877-5f79d75cd6dd None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:50.104968 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4884f-98fc-4f42-8041-2ba9614a26ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:50.105571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4884f-98fc-4f42-8041-2ba9614a26ca service cinder] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:50.105851 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4884f-98fc-4f42-8041-2ba9614a26ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:50.106124 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4884f-98fc-4f42-8041-2ba9614a26ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:50.164274 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc4884f-98fc-4f42-8041-2ba9614a26ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:50.165762 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc4884f-98fc-4f42-8041-2ba9614a26ca service cinder] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:50.174849 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1399/2775] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:54.261031 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39703ae2-a9ad-435f-ad43-82c6b8515acc service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:54.262526 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-39703ae2-a9ad-435f-ad43-82c6b8515acc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:54.322468 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39703ae2-a9ad-435f-ad43-82c6b8515acc service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:54.326044 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040bfe21-cd35-475e-83d8-e413dd0736c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:54.326660 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bfe21-cd35-475e-83d8-e413dd0736c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:54.326941 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bfe21-cd35-475e-83d8-e413dd0736c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:54.327218 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bfe21-cd35-475e-83d8-e413dd0736c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:54.328985 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-040bfe21-cd35-475e-83d8-e413dd0736c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:54.410000 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:54.414344 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040bfe21-cd35-475e-83d8-e413dd0736c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:54.416890 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040bfe21-cd35-475e-83d8-e413dd0736c4 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:54.438889 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1377/2776] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:54 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:21:55.946701 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4884f-98fc-4f42-8041-2ba9614a26ca service cinder] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:56.036332 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4884f-98fc-4f42-8041-2ba9614a26ca service cinder] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:56.041056 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced68746-7f5c-45f3-8a90-5aa2a610ed85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:56.041698 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced68746-7f5c-45f3-8a90-5aa2a610ed85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:56.042061 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced68746-7f5c-45f3-8a90-5aa2a610ed85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:56.042339 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced68746-7f5c-45f3-8a90-5aa2a610ed85 admin admin] PATH_INFO: `/v3/users/8347dd1db0cd4f02b784336c9c0cfebe` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:56.043505 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced68746-7f5c-45f3-8a90-5aa2a610ed85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8347dd1db0cd4f02b784336c9c0cfebe)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:56.045571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced68746-7f5c-45f3-8a90-5aa2a610ed85 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:56.058494 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:56.097791 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-ced68746-7f5c-45f3-8a90-5aa2a610ed85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8347dd1db0cd4f02b784336c9c0cfebe'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:56.098484 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-ced68746-7f5c-45f3-8a90-5aa2a610ed85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8347dd1db0cd4f02b784336c9c0cfebe'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:56.110942 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-ced68746-7f5c-45f3-8a90-5aa2a610ed85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8347dd1db0cd4f02b784336c9c0cfebe'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:56.118517 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-ced68746-7f5c-45f3-8a90-5aa2a610ed85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8347dd1db0cd4f02b784336c9c0cfebe'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:56.119173 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-ced68746-7f5c-45f3-8a90-5aa2a610ed85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8347dd1db0cd4f02b784336c9c0cfebe'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:56.132006 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1400/2777] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:55 2022] DELETE /identity/v3/users/8347dd1db0cd4f02b784336c9c0cfebe => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:56.136296 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e18cdd-7235-4c58-bd44-91e1b26d697c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:56.136797 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e18cdd-7235-4c58-bd44-91e1b26d697c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:56.137714 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e18cdd-7235-4c58-bd44-91e1b26d697c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:56.158147 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-47e18cdd-7235-4c58-bd44-91e1b26d697c None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:21:56.159970 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-47e18cdd-7235-4c58-bd44-91e1b26d697c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:56.227991 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1378/2778] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:21:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:21:56.238595 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced68746-7f5c-45f3-8a90-5aa2a610ed85 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:56.241121 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-ced68746-7f5c-45f3-8a90-5aa2a610ed85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:56.300435 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced68746-7f5c-45f3-8a90-5aa2a610ed85 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:56.304143 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad93b5b-f30b-43e9-af11-1de13ac4cdb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:56.304804 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad93b5b-f30b-43e9-af11-1de13ac4cdb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:56.305090 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad93b5b-f30b-43e9-af11-1de13ac4cdb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:56.305488 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad93b5b-f30b-43e9-af11-1de13ac4cdb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:56.315299 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad93b5b-f30b-43e9-af11-1de13ac4cdb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:56.317404 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad93b5b-f30b-43e9-af11-1de13ac4cdb0 service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:56.336763 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1401/2779] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:21:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:21:56.668023 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e18cdd-7235-4c58-bd44-91e1b26d697c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:56.670391 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-47e18cdd-7235-4c58-bd44-91e1b26d697c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:56.695974 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e18cdd-7235-4c58-bd44-91e1b26d697c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:56.701641 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54459bf5-4c14-4ac9-83a1-056ca2c38e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:56.705103 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54459bf5-4c14-4ac9-83a1-056ca2c38e92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:56.705431 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54459bf5-4c14-4ac9-83a1-056ca2c38e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:56.705840 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54459bf5-4c14-4ac9-83a1-056ca2c38e92 admin admin] PATH_INFO: `/v3/projects/8d0badb911c94fc985cc8b3e40a1f085` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:56.707041 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54459bf5-4c14-4ac9-83a1-056ca2c38e92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d0badb911c94fc985cc8b3e40a1f085)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:56.709047 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54459bf5-4c14-4ac9-83a1-056ca2c38e92 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:56.775737 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-54459bf5-4c14-4ac9-83a1-056ca2c38e92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d0badb911c94fc985cc8b3e40a1f085', 'request_id': 'req-54459bf5-4c14-4ac9-83a1-056ca2c38e92'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:56.786640 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-54459bf5-4c14-4ac9-83a1-056ca2c38e92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d0badb911c94fc985cc8b3e40a1f085', 'request_id': 'req-54459bf5-4c14-4ac9-83a1-056ca2c38e92'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:56.793590 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-54459bf5-4c14-4ac9-83a1-056ca2c38e92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d0badb911c94fc985cc8b3e40a1f085', 'request_id': 'req-54459bf5-4c14-4ac9-83a1-056ca2c38e92'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:56.809442 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-54459bf5-4c14-4ac9-83a1-056ca2c38e92 admin admin] The token cache is being invalidate because project 8d0badb911c94fc985cc8b3e40a1f085 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:56.809913 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-54459bf5-4c14-4ac9-83a1-056ca2c38e92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:56.810559 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-54459bf5-4c14-4ac9-83a1-056ca2c38e92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:56.812351 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1379/2780] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:56 2022] DELETE /identity/v3/projects/8d0badb911c94fc985cc8b3e40a1f085 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:56.817337 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad93b5b-f30b-43e9-af11-1de13ac4cdb0 service neutron] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:56.840580 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad93b5b-f30b-43e9-af11-1de13ac4cdb0 service neutron] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:56.910304 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d67739-d86c-4361-9996-3bcffb473d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:56.910963 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d67739-d86c-4361-9996-3bcffb473d37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:56.911248 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d67739-d86c-4361-9996-3bcffb473d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:56.911520 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d67739-d86c-4361-9996-3bcffb473d37 admin admin] PATH_INFO: `/v3/users/75ca572861bc460ca23a18cf2809e2a5` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:56.912775 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d67739-d86c-4361-9996-3bcffb473d37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75ca572861bc460ca23a18cf2809e2a5)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:56.914723 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d67739-d86c-4361-9996-3bcffb473d37 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:56.963235 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:56.978395 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-77d67739-d86c-4361-9996-3bcffb473d37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75ca572861bc460ca23a18cf2809e2a5'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:56.979675 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-77d67739-d86c-4361-9996-3bcffb473d37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75ca572861bc460ca23a18cf2809e2a5'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:56.996653 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-77d67739-d86c-4361-9996-3bcffb473d37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75ca572861bc460ca23a18cf2809e2a5'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:57.005321 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-77d67739-d86c-4361-9996-3bcffb473d37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75ca572861bc460ca23a18cf2809e2a5'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:57.006636 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-77d67739-d86c-4361-9996-3bcffb473d37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75ca572861bc460ca23a18cf2809e2a5'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:57.034870 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1402/2781] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:21:56 2022] DELETE /identity/v3/users/75ca572861bc460ca23a18cf2809e2a5 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:21:57.403179 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54459bf5-4c14-4ac9-83a1-056ca2c38e92 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:21:57.405214 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-54459bf5-4c14-4ac9-83a1-056ca2c38e92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:21:57.439127 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54459bf5-4c14-4ac9-83a1-056ca2c38e92 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:21:57.503768 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa0141-b4ba-41d0-ac09-ec52a3c7ba2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:21:57.504427 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa0141-b4ba-41d0-ac09-ec52a3c7ba2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:21:57.504816 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa0141-b4ba-41d0-ac09-ec52a3c7ba2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:21:57.505102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa0141-b4ba-41d0-ac09-ec52a3c7ba2a admin admin] PATH_INFO: `/v3/projects/f785cc0b629d42c6b457d0213beb23b9` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:21:57.506273 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aa0141-b4ba-41d0-ac09-ec52a3c7ba2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f785cc0b629d42c6b457d0213beb23b9)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:21:57.508216 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aa0141-b4ba-41d0-ac09-ec52a3c7ba2a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:21:57.519020 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:21:57.555797 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d2aa0141-b4ba-41d0-ac09-ec52a3c7ba2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f785cc0b629d42c6b457d0213beb23b9', 'request_id': 'req-d2aa0141-b4ba-41d0-ac09-ec52a3c7ba2a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:57.563170 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d2aa0141-b4ba-41d0-ac09-ec52a3c7ba2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f785cc0b629d42c6b457d0213beb23b9', 'request_id': 'req-d2aa0141-b4ba-41d0-ac09-ec52a3c7ba2a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:57.569852 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d2aa0141-b4ba-41d0-ac09-ec52a3c7ba2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f785cc0b629d42c6b457d0213beb23b9', 'request_id': 'req-d2aa0141-b4ba-41d0-ac09-ec52a3c7ba2a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:57.580176 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d2aa0141-b4ba-41d0-ac09-ec52a3c7ba2a admin admin] The token cache is being invalidate because project f785cc0b629d42c6b457d0213beb23b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:21:57.580471 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d2aa0141-b4ba-41d0-ac09-ec52a3c7ba2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:57.581191 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d2aa0141-b4ba-41d0-ac09-ec52a3c7ba2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:21:57.582720 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1380/2782] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:21:57 2022] DELETE /identity/v3/projects/f785cc0b629d42c6b457d0213beb23b9 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:22:07.023297 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=16,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:07.577271 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=20,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:08.146370 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d67739-d86c-4361-9996-3bcffb473d37 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:08.148852 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-77d67739-d86c-4361-9996-3bcffb473d37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:08.297127 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d67739-d86c-4361-9996-3bcffb473d37 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:08.381599 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa0141-b4ba-41d0-ac09-ec52a3c7ba2a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:08.383419 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d2aa0141-b4ba-41d0-ac09-ec52a3c7ba2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:08.434969 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f2f41-0bcc-4c4e-82e1-22a909e6c0fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:08.435623 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f2f41-0bcc-4c4e-82e1-22a909e6c0fd service nova] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:08.435916 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f2f41-0bcc-4c4e-82e1-22a909e6c0fd service nova] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:08.436202 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f2f41-0bcc-4c4e-82e1-22a909e6c0fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:08.438509 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-3d8f2f41-0bcc-4c4e-82e1-22a909e6c0fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:08.454026 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8f2f41-0bcc-4c4e-82e1-22a909e6c0fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:08.456764 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8f2f41-0bcc-4c4e-82e1-22a909e6c0fd service nova] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:08.473455 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1403/2783] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:22:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:22:08.594458 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa0141-b4ba-41d0-ac09-ec52a3c7ba2a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:08.605551 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d7d68-69ad-46d5-9bc5-a4a1aafb5af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:08.607859 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d7d68-69ad-46d5-9bc5-a4a1aafb5af4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:08.608575 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d7d68-69ad-46d5-9bc5-a4a1aafb5af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:08.609235 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d7d68-69ad-46d5-9bc5-a4a1aafb5af4 admin admin] PATH_INFO: `/v3/users/7f89506c03d34f9faecb80c073fb0add` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:08.612688 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8d7d68-69ad-46d5-9bc5-a4a1aafb5af4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f89506c03d34f9faecb80c073fb0add)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:08.617235 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8d7d68-69ad-46d5-9bc5-a4a1aafb5af4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:08.713887 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3b8d7d68-69ad-46d5-9bc5-a4a1aafb5af4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f89506c03d34f9faecb80c073fb0add'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:08.731247 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3b8d7d68-69ad-46d5-9bc5-a4a1aafb5af4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f89506c03d34f9faecb80c073fb0add'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:08.732921 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3b8d7d68-69ad-46d5-9bc5-a4a1aafb5af4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f89506c03d34f9faecb80c073fb0add'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:08.746266 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3b8d7d68-69ad-46d5-9bc5-a4a1aafb5af4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f89506c03d34f9faecb80c073fb0add'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:08.748176 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3b8d7d68-69ad-46d5-9bc5-a4a1aafb5af4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f89506c03d34f9faecb80c073fb0add'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:08.773867 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1381/2784] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:22:08 2022] DELETE /identity/v3/users/7f89506c03d34f9faecb80c073fb0add => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:22:08.778479 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e8a90-095f-423c-b532-85dd30b1c20a None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:08.778714 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e8a90-095f-423c-b532-85dd30b1c20a None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:08.779001 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e8a90-095f-423c-b532-85dd30b1c20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:08.798641 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-e67e8a90-095f-423c-b532-85dd30b1c20a None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:08.800450 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-e67e8a90-095f-423c-b532-85dd30b1c20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:08.887582 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1404/2785] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:22:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:22:08.898391 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d7d68-69ad-46d5-9bc5-a4a1aafb5af4 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:08.899891 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3b8d7d68-69ad-46d5-9bc5-a4a1aafb5af4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:08.967069 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d7d68-69ad-46d5-9bc5-a4a1aafb5af4 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:08.970993 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9ed1d-c800-4f46-b2c8-0fb0bb349e01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:08.971669 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9ed1d-c800-4f46-b2c8-0fb0bb349e01 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:08.971966 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9ed1d-c800-4f46-b2c8-0fb0bb349e01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:08.972247 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9ed1d-c800-4f46-b2c8-0fb0bb349e01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:08.982622 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d9ed1d-c800-4f46-b2c8-0fb0bb349e01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:08.984076 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d9ed1d-c800-4f46-b2c8-0fb0bb349e01 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:08.997349 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1382/2786] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:22:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:22:09.479621 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e8a90-095f-423c-b532-85dd30b1c20a None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:09.481268 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-e67e8a90-095f-423c-b532-85dd30b1c20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:09.502418 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e8a90-095f-423c-b532-85dd30b1c20a None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:09.509627 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd11672-6aae-4f4d-b15b-db7937a9e877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:09.510390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd11672-6aae-4f4d-b15b-db7937a9e877 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:09.510710 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd11672-6aae-4f4d-b15b-db7937a9e877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:09.511042 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd11672-6aae-4f4d-b15b-db7937a9e877 admin admin] PATH_INFO: `/v3/projects/3de24e2ea8e24ff2a789cccf73440188` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:09.514425 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd11672-6aae-4f4d-b15b-db7937a9e877 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3de24e2ea8e24ff2a789cccf73440188)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:09.520553 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd11672-6aae-4f4d-b15b-db7937a9e877 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:09.559151 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=100 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:09.596241 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-9fd11672-6aae-4f4d-b15b-db7937a9e877 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3de24e2ea8e24ff2a789cccf73440188', 'request_id': 'req-9fd11672-6aae-4f4d-b15b-db7937a9e877'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:09.603440 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-9fd11672-6aae-4f4d-b15b-db7937a9e877 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3de24e2ea8e24ff2a789cccf73440188', 'request_id': 'req-9fd11672-6aae-4f4d-b15b-db7937a9e877'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:09.609268 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-9fd11672-6aae-4f4d-b15b-db7937a9e877 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3de24e2ea8e24ff2a789cccf73440188', 'request_id': 'req-9fd11672-6aae-4f4d-b15b-db7937a9e877'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:09.623235 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-9fd11672-6aae-4f4d-b15b-db7937a9e877 admin admin] The token cache is being invalidate because project 3de24e2ea8e24ff2a789cccf73440188 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:22:09.623519 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-9fd11672-6aae-4f4d-b15b-db7937a9e877 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:09.624193 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-9fd11672-6aae-4f4d-b15b-db7937a9e877 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:09.625948 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1405/2787] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:22:09 2022] DELETE /identity/v3/projects/3de24e2ea8e24ff2a789cccf73440188 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:22:09.631118 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77d87d-e85a-42e9-b03b-bc3376a73d35 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:09.631634 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77d87d-e85a-42e9-b03b-bc3376a73d35 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:09.631731 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77d87d-e85a-42e9-b03b-bc3376a73d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:09.651564 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-fd77d87d-e85a-42e9-b03b-bc3376a73d35 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:09.652813 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fd77d87d-e85a-42e9-b03b-bc3376a73d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:09.670713 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:09.717048 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1383/2788] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:22:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:22:09.726494 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9016dd3-1a7a-4b81-83bc-f90fbbd6889a None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:09.726830 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9016dd3-1a7a-4b81-83bc-f90fbbd6889a None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:09.727031 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9016dd3-1a7a-4b81-83bc-f90fbbd6889a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:09.752810 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-b9016dd3-1a7a-4b81-83bc-f90fbbd6889a None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:09.754136 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-b9016dd3-1a7a-4b81-83bc-f90fbbd6889a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:09.763191 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1406/2789] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:22:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:22:09.768762 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77d87d-e85a-42e9-b03b-bc3376a73d35 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:09.780079 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77d87d-e85a-42e9-b03b-bc3376a73d35 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:09.787037 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da0f35-4a70-4a8d-90df-6cf1ebf07271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:09.787816 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da0f35-4a70-4a8d-90df-6cf1ebf07271 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:09.788101 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da0f35-4a70-4a8d-90df-6cf1ebf07271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:09.788382 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da0f35-4a70-4a8d-90df-6cf1ebf07271 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:09.789131 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12da0f35-4a70-4a8d-90df-6cf1ebf07271 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:22:09.789440 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12da0f35-4a70-4a8d-90df-6cf1ebf07271 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:09.791127 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12da0f35-4a70-4a8d-90df-6cf1ebf07271 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:09.805296 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1384/2790] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:22:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:22:09.809662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9016dd3-1a7a-4b81-83bc-f90fbbd6889a None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:09.834060 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9016dd3-1a7a-4b81-83bc-f90fbbd6889a None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:09.838420 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3088629c-659d-4ed9-a869-ef924cc5c9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:09.839726 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3088629c-659d-4ed9-a869-ef924cc5c9f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:09.840223 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3088629c-659d-4ed9-a869-ef924cc5c9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:09.840736 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3088629c-659d-4ed9-a869-ef924cc5c9f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:09.841789 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3088629c-659d-4ed9-a869-ef924cc5c9f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:09.844293 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3088629c-659d-4ed9-a869-ef924cc5c9f3 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:09.887054 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1407/2791] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:22:09 2022] POST /identity/v3/projects => generated 373 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:22:09.891192 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da0f35-4a70-4a8d-90df-6cf1ebf07271 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:09.907232 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da0f35-4a70-4a8d-90df-6cf1ebf07271 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:09.986200 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045519dc-30e8-43de-8abf-d7eeb2d10481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:09.986840 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045519dc-30e8-43de-8abf-d7eeb2d10481 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:09.987247 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045519dc-30e8-43de-8abf-d7eeb2d10481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:09.987535 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045519dc-30e8-43de-8abf-d7eeb2d10481 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:09.988228 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045519dc-30e8-43de-8abf-d7eeb2d10481 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:09.990194 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045519dc-30e8-43de-8abf-d7eeb2d10481 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:10.012856 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1385/2792] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:22:09 2022] POST /identity/v3/users => generated 287 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:22:10.017040 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3088629c-659d-4ed9-a869-ef924cc5c9f3 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:10.028086 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3088629c-659d-4ed9-a869-ef924cc5c9f3 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:10.033173 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e348509-b2f9-4222-9bf4-598f3d7e2152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:10.034016 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e348509-b2f9-4222-9bf4-598f3d7e2152 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:10.034305 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e348509-b2f9-4222-9bf4-598f3d7e2152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:10.034584 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e348509-b2f9-4222-9bf4-598f3d7e2152 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:10.035322 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e348509-b2f9-4222-9bf4-598f3d7e2152 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:10.037085 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e348509-b2f9-4222-9bf4-598f3d7e2152 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:10.047372 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1408/2793] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:22:10.051533 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045519dc-30e8-43de-8abf-d7eeb2d10481 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:10.063602 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045519dc-30e8-43de-8abf-d7eeb2d10481 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:10.069977 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826cded3-625b-4640-a1e1-2799c737558c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:10.070744 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cded3-625b-4640-a1e1-2799c737558c admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:10.071031 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cded3-625b-4640-a1e1-2799c737558c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:10.071307 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cded3-625b-4640-a1e1-2799c737558c admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:10.072012 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826cded3-625b-4640-a1e1-2799c737558c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:10.073919 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826cded3-625b-4640-a1e1-2799c737558c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:10.092981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1386/2794] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:22:10.094991 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e348509-b2f9-4222-9bf4-598f3d7e2152 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:10.113968 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e348509-b2f9-4222-9bf4-598f3d7e2152 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:10.119929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64ef36-813a-4cf5-85f7-0ffd6ede09d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:10.120694 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64ef36-813a-4cf5-85f7-0ffd6ede09d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:10.121016 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64ef36-813a-4cf5-85f7-0ffd6ede09d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:10.121330 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64ef36-813a-4cf5-85f7-0ffd6ede09d5 admin admin] PATH_INFO: `/v3/projects/d0a35088cde24e1ba11554e054faa959/users/948d2201db024e1da10c00a0606dba65/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:10.144828 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db64ef36-813a-4cf5-85f7-0ffd6ede09d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0a35088cde24e1ba11554e054faa959, user_id=948d2201db024e1da10c00a0606dba65, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:10.147740 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db64ef36-813a-4cf5-85f7-0ffd6ede09d5 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:10.162028 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1409/2795] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:22:10 2022] PUT /identity/v3/projects/d0a35088cde24e1ba11554e054faa959/users/948d2201db024e1da10c00a0606dba65/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:22:10.173249 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff262ce-dfcb-4761-9614-0e21caecc00d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:10.173881 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff262ce-dfcb-4761-9614-0e21caecc00d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:10.174076 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff262ce-dfcb-4761-9614-0e21caecc00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:10.208123 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-1ff262ce-dfcb-4761-9614-0e21caecc00d None None] MFA Rules not processed for user `948d2201db024e1da10c00a0606dba65`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:10.210065 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1ff262ce-dfcb-4761-9614-0e21caecc00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:10.279108 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1387/2796] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:22:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:22:10.290492 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64ef36-813a-4cf5-85f7-0ffd6ede09d5 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:10.292283 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-db64ef36-813a-4cf5-85f7-0ffd6ede09d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:10.358063 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64ef36-813a-4cf5-85f7-0ffd6ede09d5 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:10.365913 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c53da53-12da-472e-9608-7265711611f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:10.367882 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c53da53-12da-472e-9608-7265711611f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:10.368837 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c53da53-12da-472e-9608-7265711611f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:10.369311 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c53da53-12da-472e-9608-7265711611f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:10.382943 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c53da53-12da-472e-9608-7265711611f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:10.384556 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c53da53-12da-472e-9608-7265711611f4 service cinder] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:10.397353 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1410/2797] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:22:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:22:11.123014 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff262ce-dfcb-4761-9614-0e21caecc00d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:11.124520 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1ff262ce-dfcb-4761-9614-0e21caecc00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:11.147742 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:11.197807 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff262ce-dfcb-4761-9614-0e21caecc00d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:11.202171 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c46a1-c919-411b-8917-8b8fdb39e19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:11.202843 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c46a1-c919-411b-8917-8b8fdb39e19d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:11.203136 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c46a1-c919-411b-8917-8b8fdb39e19d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:11.203415 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c46a1-c919-411b-8917-8b8fdb39e19d admin admin] PATH_INFO: `/v3/users/b15e34f618764704bc876d10216cbf82` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:11.204624 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9c46a1-c919-411b-8917-8b8fdb39e19d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b15e34f618764704bc876d10216cbf82)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:11.207129 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9c46a1-c919-411b-8917-8b8fdb39e19d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:11.255941 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0c9c46a1-c919-411b-8917-8b8fdb39e19d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b15e34f618764704bc876d10216cbf82'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:11.263254 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0c9c46a1-c919-411b-8917-8b8fdb39e19d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b15e34f618764704bc876d10216cbf82'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:11.263925 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0c9c46a1-c919-411b-8917-8b8fdb39e19d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b15e34f618764704bc876d10216cbf82'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:11.272272 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0c9c46a1-c919-411b-8917-8b8fdb39e19d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b15e34f618764704bc876d10216cbf82'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:11.272940 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0c9c46a1-c919-411b-8917-8b8fdb39e19d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b15e34f618764704bc876d10216cbf82'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:11.290912 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1388/2798] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:22:11 2022] DELETE /identity/v3/users/b15e34f618764704bc876d10216cbf82 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:22:11.471716 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c53da53-12da-472e-9608-7265711611f4 service cinder] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:11.486034 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c53da53-12da-472e-9608-7265711611f4 service cinder] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:11.557142 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:11.571608 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c924eaf-1bed-482a-aa5e-96145d1209c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:11.572250 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c924eaf-1bed-482a-aa5e-96145d1209c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:11.572560 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c924eaf-1bed-482a-aa5e-96145d1209c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:11.573390 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c924eaf-1bed-482a-aa5e-96145d1209c1 admin admin] PATH_INFO: `/v3/projects/de5aeb731d2c409189934d5845a1fcd8` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:11.575374 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c924eaf-1bed-482a-aa5e-96145d1209c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de5aeb731d2c409189934d5845a1fcd8)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:11.577304 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c924eaf-1bed-482a-aa5e-96145d1209c1 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:11.628382 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-9c924eaf-1bed-482a-aa5e-96145d1209c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de5aeb731d2c409189934d5845a1fcd8', 'request_id': 'req-9c924eaf-1bed-482a-aa5e-96145d1209c1'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:11.635235 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-9c924eaf-1bed-482a-aa5e-96145d1209c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de5aeb731d2c409189934d5845a1fcd8', 'request_id': 'req-9c924eaf-1bed-482a-aa5e-96145d1209c1'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:11.641773 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-9c924eaf-1bed-482a-aa5e-96145d1209c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de5aeb731d2c409189934d5845a1fcd8', 'request_id': 'req-9c924eaf-1bed-482a-aa5e-96145d1209c1'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:11.662624 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-9c924eaf-1bed-482a-aa5e-96145d1209c1 admin admin] The token cache is being invalidate because project de5aeb731d2c409189934d5845a1fcd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:22:11.662921 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-9c924eaf-1bed-482a-aa5e-96145d1209c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:11.663545 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-9c924eaf-1bed-482a-aa5e-96145d1209c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:11.665083 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1411/2799] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:22:11 2022] DELETE /identity/v3/projects/de5aeb731d2c409189934d5845a1fcd8 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:22:11.669114 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c46a1-c919-411b-8917-8b8fdb39e19d admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:11.670523 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0c9c46a1-c919-411b-8917-8b8fdb39e19d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:11.683479 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c46a1-c919-411b-8917-8b8fdb39e19d admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:11.738822 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232584f7-d959-49f7-9547-da71efa1b17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:11.739457 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232584f7-d959-49f7-9547-da71efa1b17e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:11.739738 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232584f7-d959-49f7-9547-da71efa1b17e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:11.740013 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232584f7-d959-49f7-9547-da71efa1b17e admin admin] PATH_INFO: `/v3/users/77e055fa535545ef924a95b611dfb713` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:11.741184 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232584f7-d959-49f7-9547-da71efa1b17e admin admin] RBAC: Authorizing `identity:delete_user(user_id=77e055fa535545ef924a95b611dfb713)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:11.743077 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232584f7-d959-49f7-9547-da71efa1b17e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:11.786154 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:11.793765 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-232584f7-d959-49f7-9547-da71efa1b17e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77e055fa535545ef924a95b611dfb713'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:11.805578 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-232584f7-d959-49f7-9547-da71efa1b17e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77e055fa535545ef924a95b611dfb713'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:11.806403 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-232584f7-d959-49f7-9547-da71efa1b17e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77e055fa535545ef924a95b611dfb713'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:11.819660 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-232584f7-d959-49f7-9547-da71efa1b17e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77e055fa535545ef924a95b611dfb713'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:11.820544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-232584f7-d959-49f7-9547-da71efa1b17e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77e055fa535545ef924a95b611dfb713'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:11.843155 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1389/2800] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:22:11 2022] DELETE /identity/v3/users/77e055fa535545ef924a95b611dfb713 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:22:12.027201 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c924eaf-1bed-482a-aa5e-96145d1209c1 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:12.029279 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-9c924eaf-1bed-482a-aa5e-96145d1209c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:12.049269 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c924eaf-1bed-482a-aa5e-96145d1209c1 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:12.131360 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9053f4f9-8962-4a5f-abf7-166434b53dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:12.132163 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053f4f9-8962-4a5f-abf7-166434b53dd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:12.132943 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053f4f9-8962-4a5f-abf7-166434b53dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:12.132943 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053f4f9-8962-4a5f-abf7-166434b53dd5 admin admin] PATH_INFO: `/v3/projects/9df74a9b9fe14af0b9374b365f94c8bc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:12.134449 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9053f4f9-8962-4a5f-abf7-166434b53dd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9df74a9b9fe14af0b9374b365f94c8bc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:12.137152 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9053f4f9-8962-4a5f-abf7-166434b53dd5 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:12.201559 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-9053f4f9-8962-4a5f-abf7-166434b53dd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9df74a9b9fe14af0b9374b365f94c8bc', 'request_id': 'req-9053f4f9-8962-4a5f-abf7-166434b53dd5'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:12.208210 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-9053f4f9-8962-4a5f-abf7-166434b53dd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9df74a9b9fe14af0b9374b365f94c8bc', 'request_id': 'req-9053f4f9-8962-4a5f-abf7-166434b53dd5'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:12.214362 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-9053f4f9-8962-4a5f-abf7-166434b53dd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9df74a9b9fe14af0b9374b365f94c8bc', 'request_id': 'req-9053f4f9-8962-4a5f-abf7-166434b53dd5'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:12.227708 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-9053f4f9-8962-4a5f-abf7-166434b53dd5 admin admin] The token cache is being invalidate because project 9df74a9b9fe14af0b9374b365f94c8bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:22:12.228020 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-9053f4f9-8962-4a5f-abf7-166434b53dd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:12.228706 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-9053f4f9-8962-4a5f-abf7-166434b53dd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:12.230575 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1412/2801] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:22:12 2022] DELETE /identity/v3/projects/9df74a9b9fe14af0b9374b365f94c8bc => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:22:12.235635 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a87c9-46f9-4fb9-96c9-ebf5c09efca8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:12.236727 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a87c9-46f9-4fb9-96c9-ebf5c09efca8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:12.236871 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a87c9-46f9-4fb9-96c9-ebf5c09efca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:12.260630 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-992a87c9-46f9-4fb9-96c9-ebf5c09efca8 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:12.261822 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-992a87c9-46f9-4fb9-96c9-ebf5c09efca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:12.325485 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1390/2802] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:22:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:22:12.332989 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1aa3f-b71e-4aad-9935-31cf99dae13f None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:12.333281 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1aa3f-b71e-4aad-9935-31cf99dae13f None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:12.333618 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1aa3f-b71e-4aad-9935-31cf99dae13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:12.348909 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:12.355463 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-15d1aa3f-b71e-4aad-9935-31cf99dae13f None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:12.357408 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-15d1aa3f-b71e-4aad-9935-31cf99dae13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:12.367088 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1413/2803] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:22:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:22:12.371862 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a87c9-46f9-4fb9-96c9-ebf5c09efca8 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:12.382438 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a87c9-46f9-4fb9-96c9-ebf5c09efca8 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:12.387547 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0a0fb-5414-4db5-935a-c38761a15a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:12.388237 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0a0fb-5414-4db5-935a-c38761a15a5f admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:12.388630 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0a0fb-5414-4db5-935a-c38761a15a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:12.388980 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0a0fb-5414-4db5-935a-c38761a15a5f admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:12.389688 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c0a0fb-5414-4db5-935a-c38761a15a5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:22:12.389995 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c0a0fb-5414-4db5-935a-c38761a15a5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:12.391700 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c0a0fb-5414-4db5-935a-c38761a15a5f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:12.404510 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1391/2804] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:22:12.408912 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1aa3f-b71e-4aad-9935-31cf99dae13f None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:12.419233 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1aa3f-b71e-4aad-9935-31cf99dae13f None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:12.425128 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9179aa06-728d-4439-97ad-e9e2b5a710f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:12.426120 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179aa06-728d-4439-97ad-e9e2b5a710f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:12.426480 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179aa06-728d-4439-97ad-e9e2b5a710f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:12.426841 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179aa06-728d-4439-97ad-e9e2b5a710f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:12.427807 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9179aa06-728d-4439-97ad-e9e2b5a710f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:12.430430 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9179aa06-728d-4439-97ad-e9e2b5a710f5 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:12.463813 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1414/2805] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:22:12 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:22:12.465335 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0a0fb-5414-4db5-935a-c38761a15a5f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:12.477842 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0a0fb-5414-4db5-935a-c38761a15a5f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:12.539206 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195939e6-f7f2-44ec-bad2-def9c5ca9540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:12.539830 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195939e6-f7f2-44ec-bad2-def9c5ca9540 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:12.540118 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195939e6-f7f2-44ec-bad2-def9c5ca9540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:12.540397 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195939e6-f7f2-44ec-bad2-def9c5ca9540 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:12.541086 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195939e6-f7f2-44ec-bad2-def9c5ca9540 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:12.543000 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195939e6-f7f2-44ec-bad2-def9c5ca9540 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:12.569193 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1392/2806] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:22:12 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:22:12.570001 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9179aa06-728d-4439-97ad-e9e2b5a710f5 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:12.581432 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9179aa06-728d-4439-97ad-e9e2b5a710f5 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:12.586282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e35ad1-eaa9-434b-9200-d37daa452cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:12.586938 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e35ad1-eaa9-434b-9200-d37daa452cb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:12.587177 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e35ad1-eaa9-434b-9200-d37daa452cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:12.587460 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e35ad1-eaa9-434b-9200-d37daa452cb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:12.588191 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e35ad1-eaa9-434b-9200-d37daa452cb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:12.589891 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e35ad1-eaa9-434b-9200-d37daa452cb4 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:12.600824 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1415/2807] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:22:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:22:12.605046 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195939e6-f7f2-44ec-bad2-def9c5ca9540 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:12.615772 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195939e6-f7f2-44ec-bad2-def9c5ca9540 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:12.620418 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303495c4-be60-476d-ab8f-1ae9bdbc8379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:12.621063 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303495c4-be60-476d-ab8f-1ae9bdbc8379 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:12.621348 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303495c4-be60-476d-ab8f-1ae9bdbc8379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:12.621631 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303495c4-be60-476d-ab8f-1ae9bdbc8379 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:12.622320 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303495c4-be60-476d-ab8f-1ae9bdbc8379 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:12.623902 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303495c4-be60-476d-ab8f-1ae9bdbc8379 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:12.636749 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1393/2808] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:22:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:22:12.641463 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e35ad1-eaa9-434b-9200-d37daa452cb4 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:12.651415 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e35ad1-eaa9-434b-9200-d37daa452cb4 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:12.655991 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca0cd9-abb0-40ae-8d4f-86aa1460782c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:12.656994 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca0cd9-abb0-40ae-8d4f-86aa1460782c admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:12.657274 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca0cd9-abb0-40ae-8d4f-86aa1460782c admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:12.657554 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca0cd9-abb0-40ae-8d4f-86aa1460782c admin admin] PATH_INFO: `/v3/projects/7492c715a3cd4b97afc68626ee247e9c/users/f8dc9a3b792b488ea5a5ab96c7c3899d/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:12.672398 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca0cd9-abb0-40ae-8d4f-86aa1460782c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7492c715a3cd4b97afc68626ee247e9c, user_id=f8dc9a3b792b488ea5a5ab96c7c3899d, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:12.675682 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca0cd9-abb0-40ae-8d4f-86aa1460782c admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:12.690504 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1416/2809] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:22:12 2022] PUT /identity/v3/projects/7492c715a3cd4b97afc68626ee247e9c/users/f8dc9a3b792b488ea5a5ab96c7c3899d/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:22:12.761170 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866be6a-ea2c-4e32-9f9e-3a1f8e0adf70 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:12.761456 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866be6a-ea2c-4e32-9f9e-3a1f8e0adf70 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:12.761736 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866be6a-ea2c-4e32-9f9e-3a1f8e0adf70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:12.790151 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d866be6a-ea2c-4e32-9f9e-3a1f8e0adf70 None None] MFA Rules not processed for user `f8dc9a3b792b488ea5a5ab96c7c3899d`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:12.791370 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d866be6a-ea2c-4e32-9f9e-3a1f8e0adf70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:12.804973 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:12.890429 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1394/2810] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:22:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:22:12.900274 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca0cd9-abb0-40ae-8d4f-86aa1460782c admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:12.902073 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-c8ca0cd9-abb0-40ae-8d4f-86aa1460782c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:12.919971 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d866be6a-ea2c-4e32-9f9e-3a1f8e0adf70 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:12.921401 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d866be6a-ea2c-4e32-9f9e-3a1f8e0adf70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:12.961431 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca0cd9-abb0-40ae-8d4f-86aa1460782c admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:12.964994 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa44374-4c75-46a5-aa61-cb3104ebd3f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:12.965640 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa44374-4c75-46a5-aa61-cb3104ebd3f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:12.965916 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa44374-4c75-46a5-aa61-cb3104ebd3f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:12.966186 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa44374-4c75-46a5-aa61-cb3104ebd3f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:12.974979 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa44374-4c75-46a5-aa61-cb3104ebd3f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:12.976416 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa44374-4c75-46a5-aa61-cb3104ebd3f9 service cinder] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:12.980970 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d866be6a-ea2c-4e32-9f9e-3a1f8e0adf70 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:12.984702 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd21b846-6cea-4d16-90cb-8601584584c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:12.985307 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1417/2811] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:22:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:22:12.985464 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd21b846-6cea-4d16-90cb-8601584584c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:12.985602 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd21b846-6cea-4d16-90cb-8601584584c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:12.985906 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd21b846-6cea-4d16-90cb-8601584584c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:12.987725 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fd21b846-6cea-4d16-90cb-8601584584c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:13.006644 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa44374-4c75-46a5-aa61-cb3104ebd3f9 service cinder] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:13.054629 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd21b846-6cea-4d16-90cb-8601584584c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:13.058038 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd21b846-6cea-4d16-90cb-8601584584c3 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:13.067322 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1395/2812] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:22:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:22:13.076433 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa44374-4c75-46a5-aa61-cb3104ebd3f9 service cinder] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:13.081108 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6673b8-0bae-4e74-b231-8101252d2ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:13.081740 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6673b8-0bae-4e74-b231-8101252d2ebb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:13.082020 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6673b8-0bae-4e74-b231-8101252d2ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:13.082305 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6673b8-0bae-4e74-b231-8101252d2ebb admin admin] PATH_INFO: `/v3/users/f8dc9a3b792b488ea5a5ab96c7c3899d` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:13.083458 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6673b8-0bae-4e74-b231-8101252d2ebb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8dc9a3b792b488ea5a5ab96c7c3899d)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:13.085405 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6673b8-0bae-4e74-b231-8101252d2ebb admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:13.134910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-8c6673b8-0bae-4e74-b231-8101252d2ebb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8dc9a3b792b488ea5a5ab96c7c3899d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:13.135595 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-8c6673b8-0bae-4e74-b231-8101252d2ebb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8dc9a3b792b488ea5a5ab96c7c3899d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:13.136892 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:13.150544 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-8c6673b8-0bae-4e74-b231-8101252d2ebb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8dc9a3b792b488ea5a5ab96c7c3899d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:13.158970 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-8c6673b8-0bae-4e74-b231-8101252d2ebb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8dc9a3b792b488ea5a5ab96c7c3899d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:13.159582 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-8c6673b8-0bae-4e74-b231-8101252d2ebb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8dc9a3b792b488ea5a5ab96c7c3899d'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:13.172052 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1418/2813] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:22:13 2022] DELETE /identity/v3/users/f8dc9a3b792b488ea5a5ab96c7c3899d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:22:13.177720 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b95ef-2690-4921-a161-dce1ed264c06 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:13.178013 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b95ef-2690-4921-a161-dce1ed264c06 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:13.178320 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b95ef-2690-4921-a161-dce1ed264c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:13.198001 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-804b95ef-2690-4921-a161-dce1ed264c06 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:13.199227 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-804b95ef-2690-4921-a161-dce1ed264c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:13.262500 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1396/2814] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:22:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:22:13.269965 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6673b8-0bae-4e74-b231-8101252d2ebb admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:13.271819 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-8c6673b8-0bae-4e74-b231-8101252d2ebb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:13.339044 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6673b8-0bae-4e74-b231-8101252d2ebb admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:13.342948 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e60537-8c7b-4147-a8d5-919509c0a269 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:13.344014 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e60537-8c7b-4147-a8d5-919509c0a269 service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:13.344345 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e60537-8c7b-4147-a8d5-919509c0a269 service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:13.344777 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e60537-8c7b-4147-a8d5-919509c0a269 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:13.356215 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e60537-8c7b-4147-a8d5-919509c0a269 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:13.357956 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e60537-8c7b-4147-a8d5-919509c0a269 service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:13.367884 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1419/2815] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:22:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:22:13.515967 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b95ef-2690-4921-a161-dce1ed264c06 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:13.518164 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-804b95ef-2690-4921-a161-dce1ed264c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:13.526028 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:13.534846 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b95ef-2690-4921-a161-dce1ed264c06 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:13.539874 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e559145-4424-41c5-b6e5-3d65443c1660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:13.541133 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e559145-4424-41c5-b6e5-3d65443c1660 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:13.541544 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e559145-4424-41c5-b6e5-3d65443c1660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:13.541780 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e559145-4424-41c5-b6e5-3d65443c1660 admin admin] PATH_INFO: `/v3/users/948d2201db024e1da10c00a0606dba65` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:13.543105 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e559145-4424-41c5-b6e5-3d65443c1660 admin admin] RBAC: Authorizing `identity:delete_user(user_id=948d2201db024e1da10c00a0606dba65)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:13.545219 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e559145-4424-41c5-b6e5-3d65443c1660 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:13.603462 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4e559145-4424-41c5-b6e5-3d65443c1660 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '948d2201db024e1da10c00a0606dba65'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:13.610713 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4e559145-4424-41c5-b6e5-3d65443c1660 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '948d2201db024e1da10c00a0606dba65'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:13.611433 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4e559145-4424-41c5-b6e5-3d65443c1660 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '948d2201db024e1da10c00a0606dba65'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:13.619404 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4e559145-4424-41c5-b6e5-3d65443c1660 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '948d2201db024e1da10c00a0606dba65'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:13.620147 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4e559145-4424-41c5-b6e5-3d65443c1660 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '948d2201db024e1da10c00a0606dba65'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:13.643315 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1397/2816] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:22:13 2022] DELETE /identity/v3/users/948d2201db024e1da10c00a0606dba65 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:22:13.646899 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca059abe-2d6b-4d94-82b0-b2510d4a4552 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:13.647478 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca059abe-2d6b-4d94-82b0-b2510d4a4552 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:13.647637 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca059abe-2d6b-4d94-82b0-b2510d4a4552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:13.664305 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e559145-4424-41c5-b6e5-3d65443c1660 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:13.666055 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4e559145-4424-41c5-b6e5-3d65443c1660 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:13.668421 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-ca059abe-2d6b-4d94-82b0-b2510d4a4552 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:13.669828 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-ca059abe-2d6b-4d94-82b0-b2510d4a4552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:13.748856 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1420/2817] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:22:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:22:13.753235 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e559145-4424-41c5-b6e5-3d65443c1660 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:13.759309 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630ac937-2704-486c-9880-b00162c914ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:13.759693 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca059abe-2d6b-4d94-82b0-b2510d4a4552 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:13.759980 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ac937-2704-486c-9880-b00162c914ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:13.760265 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ac937-2704-486c-9880-b00162c914ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:13.760577 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ac937-2704-486c-9880-b00162c914ef admin admin] PATH_INFO: `/v3/projects/7492c715a3cd4b97afc68626ee247e9c` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:13.761483 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-ca059abe-2d6b-4d94-82b0-b2510d4a4552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:13.761753 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630ac937-2704-486c-9880-b00162c914ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=7492c715a3cd4b97afc68626ee247e9c)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:13.763831 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630ac937-2704-486c-9880-b00162c914ef admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:13.828453 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca059abe-2d6b-4d94-82b0-b2510d4a4552 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:13.831607 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-630ac937-2704-486c-9880-b00162c914ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7492c715a3cd4b97afc68626ee247e9c', 'request_id': 'req-630ac937-2704-486c-9880-b00162c914ef'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:13.839981 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-630ac937-2704-486c-9880-b00162c914ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7492c715a3cd4b97afc68626ee247e9c', 'request_id': 'req-630ac937-2704-486c-9880-b00162c914ef'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:13.847165 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-630ac937-2704-486c-9880-b00162c914ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7492c715a3cd4b97afc68626ee247e9c', 'request_id': 'req-630ac937-2704-486c-9880-b00162c914ef'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:13.852369 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:13.858899 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-630ac937-2704-486c-9880-b00162c914ef admin admin] The token cache is being invalidate because project 7492c715a3cd4b97afc68626ee247e9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:22:13.859098 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-630ac937-2704-486c-9880-b00162c914ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:13.859694 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-630ac937-2704-486c-9880-b00162c914ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:13.861500 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1398/2818] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:22:13 2022] DELETE /identity/v3/projects/7492c715a3cd4b97afc68626ee247e9c => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:22:13.867689 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35aaa5-8726-4775-b68f-be23b07e45f1 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:13.867985 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35aaa5-8726-4775-b68f-be23b07e45f1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:13.868548 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35aaa5-8726-4775-b68f-be23b07e45f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:13.881892 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:13.892636 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f45d60-4161-44bd-be57-766d74f2ba78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:13.893324 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f45d60-4161-44bd-be57-766d74f2ba78 service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:13.893545 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f45d60-4161-44bd-be57-766d74f2ba78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:13.893821 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f45d60-4161-44bd-be57-766d74f2ba78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:13.895952 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-05f45d60-4161-44bd-be57-766d74f2ba78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:13.897206 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-0c35aaa5-8726-4775-b68f-be23b07e45f1 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:13.899491 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0c35aaa5-8726-4775-b68f-be23b07e45f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:13.967589 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f45d60-4161-44bd-be57-766d74f2ba78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:13.969188 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f45d60-4161-44bd-be57-766d74f2ba78 service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:13.974114 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1399/2819] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:22:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:22:13.981741 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8952d76-f395-4512-a62b-0760470d4af4 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:13.982155 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8952d76-f395-4512-a62b-0760470d4af4 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:13.982527 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8952d76-f395-4512-a62b-0760470d4af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:13.988087 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1421/2820] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:22:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:22:14.017894 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c8952d76-f395-4512-a62b-0760470d4af4 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:14.019509 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c8952d76-f395-4512-a62b-0760470d4af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:14.030194 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1400/2821] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:22:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:22:14.034925 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f45d60-4161-44bd-be57-766d74f2ba78 service neutron] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:14.045337 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f45d60-4161-44bd-be57-766d74f2ba78 service neutron] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:14.053594 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fbbc40-d0c6-4c8e-847b-e1558b9035fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:14.054290 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbbc40-d0c6-4c8e-847b-e1558b9035fd admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:14.054628 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbbc40-d0c6-4c8e-847b-e1558b9035fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:14.055269 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbbc40-d0c6-4c8e-847b-e1558b9035fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:14.055865 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fbbc40-d0c6-4c8e-847b-e1558b9035fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:22:14.056177 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fbbc40-d0c6-4c8e-847b-e1558b9035fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:14.058275 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fbbc40-d0c6-4c8e-847b-e1558b9035fd admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:14.075305 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1422/2822] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:22:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:22:14.081048 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8952d76-f395-4512-a62b-0760470d4af4 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:14.092223 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8952d76-f395-4512-a62b-0760470d4af4 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:14.097531 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d489ef0-4eb9-41c4-959a-f9a9b2c9c88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:14.098237 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d489ef0-4eb9-41c4-959a-f9a9b2c9c88b admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:14.098527 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d489ef0-4eb9-41c4-959a-f9a9b2c9c88b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:14.098826 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d489ef0-4eb9-41c4-959a-f9a9b2c9c88b admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:14.099568 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d489ef0-4eb9-41c4-959a-f9a9b2c9c88b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:14.101559 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d489ef0-4eb9-41c4-959a-f9a9b2c9c88b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:14.133258 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1401/2823] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:22:14 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:22:14.137540 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fbbc40-d0c6-4c8e-847b-e1558b9035fd admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:14.152785 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fbbc40-d0c6-4c8e-847b-e1558b9035fd admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:14.230746 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13243fa-3caa-4baa-b99a-df9d6f190f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:14.231404 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13243fa-3caa-4baa-b99a-df9d6f190f84 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:14.231690 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13243fa-3caa-4baa-b99a-df9d6f190f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:14.231973 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13243fa-3caa-4baa-b99a-df9d6f190f84 admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:14.232754 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13243fa-3caa-4baa-b99a-df9d6f190f84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:14.235110 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13243fa-3caa-4baa-b99a-df9d6f190f84 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:14.265273 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1423/2824] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:22:14 2022] POST /identity/v3/users => generated 292 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:22:14.266528 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d489ef0-4eb9-41c4-959a-f9a9b2c9c88b admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:14.279009 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d489ef0-4eb9-41c4-959a-f9a9b2c9c88b admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:14.284975 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1caf8-7c38-404b-916e-8f31634df47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:14.285631 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1caf8-7c38-404b-916e-8f31634df47d admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:14.285932 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1caf8-7c38-404b-916e-8f31634df47d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:14.286199 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1caf8-7c38-404b-916e-8f31634df47d admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:14.287128 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a1caf8-7c38-404b-916e-8f31634df47d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:14.289094 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a1caf8-7c38-404b-916e-8f31634df47d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:14.301932 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1402/2825] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:22:14.305880 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13243fa-3caa-4baa-b99a-df9d6f190f84 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:14.318175 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13243fa-3caa-4baa-b99a-df9d6f190f84 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:14.323357 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f1edd-03ad-4228-a744-5d1aff488c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:14.324247 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f1edd-03ad-4228-a744-5d1aff488c57 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:14.324247 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f1edd-03ad-4228-a744-5d1aff488c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:14.324585 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f1edd-03ad-4228-a744-5d1aff488c57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:14.325829 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f1edd-03ad-4228-a744-5d1aff488c57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:14.327152 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f1edd-03ad-4228-a744-5d1aff488c57 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:14.341742 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1424/2826] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:22:14.346582 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1caf8-7c38-404b-916e-8f31634df47d admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:14.359207 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1caf8-7c38-404b-916e-8f31634df47d admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:14.359345 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f1edd-03ad-4228-a744-5d1aff488c57 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:14.361471 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-9f3f1edd-03ad-4228-a744-5d1aff488c57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:14.364973 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2674ea96-2e97-4823-8f38-248fa203827b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:14.365710 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674ea96-2e97-4823-8f38-248fa203827b admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:14.366070 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674ea96-2e97-4823-8f38-248fa203827b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:14.366424 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674ea96-2e97-4823-8f38-248fa203827b admin admin] PATH_INFO: `/v3/projects/1b86711947064f35bc72e8f352ee33b4/users/b57f5d33ff234d2092b47148eb58e4d4/roles/926b9f36c24847c582c84af904250abc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:14.379262 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f1edd-03ad-4228-a744-5d1aff488c57 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:14.384725 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335140e-b64d-4746-81e0-20c55a281aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:14.385405 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335140e-b64d-4746-81e0-20c55a281aff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:14.385689 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335140e-b64d-4746-81e0-20c55a281aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:14.386047 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335140e-b64d-4746-81e0-20c55a281aff admin admin] PATH_INFO: `/v3/projects/d0a35088cde24e1ba11554e054faa959` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:14.387638 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1335140e-b64d-4746-81e0-20c55a281aff admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0a35088cde24e1ba11554e054faa959)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:14.389720 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1335140e-b64d-4746-81e0-20c55a281aff admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:14.393146 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2674ea96-2e97-4823-8f38-248fa203827b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b86711947064f35bc72e8f352ee33b4, user_id=b57f5d33ff234d2092b47148eb58e4d4, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:14.404984 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2674ea96-2e97-4823-8f38-248fa203827b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:14.424871 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1403/2827] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:22:14 2022] PUT /identity/v3/projects/1b86711947064f35bc72e8f352ee33b4/users/b57f5d33ff234d2092b47148eb58e4d4/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:22:14.431807 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7108b0b2-216c-4f7b-8705-ec81bf8fd100 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:14.432152 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7108b0b2-216c-4f7b-8705-ec81bf8fd100 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:14.432873 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7108b0b2-216c-4f7b-8705-ec81bf8fd100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:14.441157 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:14.455347 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-1335140e-b64d-4746-81e0-20c55a281aff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0a35088cde24e1ba11554e054faa959', 'request_id': 'req-1335140e-b64d-4746-81e0-20c55a281aff'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:14.462796 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-1335140e-b64d-4746-81e0-20c55a281aff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0a35088cde24e1ba11554e054faa959', 'request_id': 'req-1335140e-b64d-4746-81e0-20c55a281aff'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:14.469026 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-1335140e-b64d-4746-81e0-20c55a281aff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0a35088cde24e1ba11554e054faa959', 'request_id': 'req-1335140e-b64d-4746-81e0-20c55a281aff'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:14.471370 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-7108b0b2-216c-4f7b-8705-ec81bf8fd100 None None] MFA Rules not processed for user `b57f5d33ff234d2092b47148eb58e4d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:14.473100 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7108b0b2-216c-4f7b-8705-ec81bf8fd100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:14.484439 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-1335140e-b64d-4746-81e0-20c55a281aff admin admin] The token cache is being invalidate because project d0a35088cde24e1ba11554e054faa959 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:22:14.484996 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-1335140e-b64d-4746-81e0-20c55a281aff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:14.486138 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-1335140e-b64d-4746-81e0-20c55a281aff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:14.487834 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1425/2828] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:22:14 2022] DELETE /identity/v3/projects/d0a35088cde24e1ba11554e054faa959 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:22:14.492652 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1bc6f-86d2-4abf-a449-f7d2e69cb844 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:14.492946 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1bc6f-86d2-4abf-a449-f7d2e69cb844 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:14.493237 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1bc6f-86d2-4abf-a449-f7d2e69cb844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:14.520703 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-b3b1bc6f-86d2-4abf-a449-f7d2e69cb844 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:14.522461 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-b3b1bc6f-86d2-4abf-a449-f7d2e69cb844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:14.551327 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1404/2829] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:22:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:22:14.552437 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7108b0b2-216c-4f7b-8705-ec81bf8fd100 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:14.553935 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7108b0b2-216c-4f7b-8705-ec81bf8fd100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:14.602088 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:14.604460 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1426/2830] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:22:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:22:14.613099 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b42b9-3111-4904-8fe3-dd3d670fb733 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:14.613384 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b42b9-3111-4904-8fe3-dd3d670fb733 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:14.613667 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b42b9-3111-4904-8fe3-dd3d670fb733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:14.634191 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-0a4b42b9-3111-4904-8fe3-dd3d670fb733 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:14.635516 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-0a4b42b9-3111-4904-8fe3-dd3d670fb733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:14.638991 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7108b0b2-216c-4f7b-8705-ec81bf8fd100 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:14.647386 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1427/2831] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:22:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:22:14.647896 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d281c9b-8336-471d-806a-94692afdac19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:14.647896 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d281c9b-8336-471d-806a-94692afdac19 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:14.647896 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d281c9b-8336-471d-806a-94692afdac19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:14.647896 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d281c9b-8336-471d-806a-94692afdac19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:14.648259 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d281c9b-8336-471d-806a-94692afdac19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:14.648259 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d281c9b-8336-471d-806a-94692afdac19 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:14.648407 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b42b9-3111-4904-8fe3-dd3d670fb733 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:14.659202 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b42b9-3111-4904-8fe3-dd3d670fb733 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:14.665075 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ad9fd-251e-4b7c-ab40-30b4d1ea1f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:14.665813 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ad9fd-251e-4b7c-ab40-30b4d1ea1f22 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:14.666097 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ad9fd-251e-4b7c-ab40-30b4d1ea1f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:14.666496 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ad9fd-251e-4b7c-ab40-30b4d1ea1f22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:14.667384 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5ad9fd-251e-4b7c-ab40-30b4d1ea1f22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:22:14.667733 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5ad9fd-251e-4b7c-ab40-30b4d1ea1f22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:14.669549 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5ad9fd-251e-4b7c-ab40-30b4d1ea1f22 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:14.678315 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1405/2832] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:22:14 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:22:14.682483 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d281c9b-8336-471d-806a-94692afdac19 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:14.683215 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1428/2833] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:22:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:22:14.687267 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ad9fd-251e-4b7c-ab40-30b4d1ea1f22 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:14.698867 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ad9fd-251e-4b7c-ab40-30b4d1ea1f22 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:14.763292 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d281c9b-8336-471d-806a-94692afdac19 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:14.773399 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bce6a5-8de7-4bef-8dec-f1d6d3cace3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:14.774672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bce6a5-8de7-4bef-8dec-f1d6d3cace3b admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:14.774672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bce6a5-8de7-4bef-8dec-f1d6d3cace3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:14.774672 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bce6a5-8de7-4bef-8dec-f1d6d3cace3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:14.775413 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bce6a5-8de7-4bef-8dec-f1d6d3cace3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:14.776987 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bdaec7-91a6-4078-9220-a8d53c5d2179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:14.777542 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bce6a5-8de7-4bef-8dec-f1d6d3cace3b admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:14.777891 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bdaec7-91a6-4078-9220-a8d53c5d2179 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:14.778682 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bdaec7-91a6-4078-9220-a8d53c5d2179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:14.778682 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bdaec7-91a6-4078-9220-a8d53c5d2179 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:14.779645 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bdaec7-91a6-4078-9220-a8d53c5d2179 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:14.781845 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bdaec7-91a6-4078-9220-a8d53c5d2179 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:14.813657 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1406/2834] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:22:14 2022] POST /identity/v3/users => generated 293 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:22:14.821410 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bdaec7-91a6-4078-9220-a8d53c5d2179 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:14.823791 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1429/2835] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:22:14 2022] POST /identity/v3/projects => generated 363 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:22:14.828361 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bce6a5-8de7-4bef-8dec-f1d6d3cace3b admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:14.832848 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bdaec7-91a6-4078-9220-a8d53c5d2179 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:14.874393 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:14.905550 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6725ff76-e293-426f-ad6d-2bef158fdadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:14.906177 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725ff76-e293-426f-ad6d-2bef158fdadb admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:14.906458 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725ff76-e293-426f-ad6d-2bef158fdadb admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:14.907058 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725ff76-e293-426f-ad6d-2bef158fdadb admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:14.909238 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6725ff76-e293-426f-ad6d-2bef158fdadb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:14.911556 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6725ff76-e293-426f-ad6d-2bef158fdadb admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:14.924502 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1407/2836] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:22:14.927912 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bce6a5-8de7-4bef-8dec-f1d6d3cace3b admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:14.928774 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6725ff76-e293-426f-ad6d-2bef158fdadb admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:14.936701 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1396106e-bc10-4f7d-aa09-4386861ca198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:14.938209 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396106e-bc10-4f7d-aa09-4386861ca198 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:14.938851 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396106e-bc10-4f7d-aa09-4386861ca198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:14.939502 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396106e-bc10-4f7d-aa09-4386861ca198 admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:14.940128 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6725ff76-e293-426f-ad6d-2bef158fdadb admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:14.940692 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1396106e-bc10-4f7d-aa09-4386861ca198 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:14.943703 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1396106e-bc10-4f7d-aa09-4386861ca198 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:14.955412 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8b8597-238e-4c6f-908e-23a8eb87d446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:14.956100 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8b8597-238e-4c6f-908e-23a8eb87d446 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:14.956403 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8b8597-238e-4c6f-908e-23a8eb87d446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:14.956687 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8b8597-238e-4c6f-908e-23a8eb87d446 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:14.957582 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8b8597-238e-4c6f-908e-23a8eb87d446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:14.960168 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8b8597-238e-4c6f-908e-23a8eb87d446 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:14.982301 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1408/2837] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:22:14.982511 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1430/2838] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:22:14 2022] POST /identity/v3/users => generated 282 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:22:14.982624 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8b8597-238e-4c6f-908e-23a8eb87d446 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:14.984016 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1396106e-bc10-4f7d-aa09-4386861ca198 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:14.999808 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8b8597-238e-4c6f-908e-23a8eb87d446 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:15.001027 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1396106e-bc10-4f7d-aa09-4386861ca198 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:15.007087 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c92a1b4-ceb5-4520-8a13-e99b66151f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:15.007851 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92a1b4-ceb5-4520-8a13-e99b66151f58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:15.008138 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92a1b4-ceb5-4520-8a13-e99b66151f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:15.008414 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92a1b4-ceb5-4520-8a13-e99b66151f58 admin admin] PATH_INFO: `/v3/projects/aec6c04dc1dd41df827993966bb8c6f0/users/b284a1110a1f4179bdd2b6071249ca03/roles/926b9f36c24847c582c84af904250abc` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:15.009043 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532b08b-ffc7-40ec-bab4-be4e0637d3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:15.009923 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532b08b-ffc7-40ec-bab4-be4e0637d3ff admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:15.011228 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532b08b-ffc7-40ec-bab4-be4e0637d3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:15.011879 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532b08b-ffc7-40ec-bab4-be4e0637d3ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:15.015365 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8532b08b-ffc7-40ec-bab4-be4e0637d3ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:15.016455 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8532b08b-ffc7-40ec-bab4-be4e0637d3ff admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:15.027869 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c92a1b4-ceb5-4520-8a13-e99b66151f58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aec6c04dc1dd41df827993966bb8c6f0, user_id=b284a1110a1f4179bdd2b6071249ca03, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:15.031073 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c92a1b4-ceb5-4520-8a13-e99b66151f58 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:15.034524 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1409/2839] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:22:15.036382 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532b08b-ffc7-40ec-bab4-be4e0637d3ff admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:15.048151 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532b08b-ffc7-40ec-bab4-be4e0637d3ff admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:15.060585 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1431/2840] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:22:14 2022] PUT /identity/v3/projects/aec6c04dc1dd41df827993966bb8c6f0/users/b284a1110a1f4179bdd2b6071249ca03/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:22:15.070309 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca6b57-32c0-435e-af19-22f4a4233dc3 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:15.070598 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca6b57-32c0-435e-af19-22f4a4233dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:15.070863 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca6b57-32c0-435e-af19-22f4a4233dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:15.075941 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:15.112423 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-40ca6b57-32c0-435e-af19-22f4a4233dc3 None None] MFA Rules not processed for user `b284a1110a1f4179bdd2b6071249ca03`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:15.113806 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-40ca6b57-32c0-435e-af19-22f4a4233dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:15.134919 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d2bba-afa9-400d-8345-cf20058f954c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:15.136087 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d2bba-afa9-400d-8345-cf20058f954c admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:15.136690 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d2bba-afa9-400d-8345-cf20058f954c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:15.137339 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d2bba-afa9-400d-8345-cf20058f954c admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:15.138895 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0d2bba-afa9-400d-8345-cf20058f954c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:15.142204 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0d2bba-afa9-400d-8345-cf20058f954c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:15.157018 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1410/2841] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:22:15.161901 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d2bba-afa9-400d-8345-cf20058f954c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:15.177264 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d2bba-afa9-400d-8345-cf20058f954c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:15.183762 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6da240-661f-40e3-a11a-7d09193e380c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:15.184688 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6da240-661f-40e3-a11a-7d09193e380c admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:15.185183 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6da240-661f-40e3-a11a-7d09193e380c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:15.185557 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6da240-661f-40e3-a11a-7d09193e380c admin admin] PATH_INFO: `/v3/projects/85760c10dd0d455ca79e274da45eddaa/users/e9fb9a6d27984900ac2ff05d609e6de2/roles/926b9f36c24847c582c84af904250abc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:15.205120 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6da240-661f-40e3-a11a-7d09193e380c admin admin] RBAC: Authorizing `identity:create_grant(project_id=85760c10dd0d455ca79e274da45eddaa, user_id=e9fb9a6d27984900ac2ff05d609e6de2, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:15.206752 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1432/2842] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:22:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:22:15.209107 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6da240-661f-40e3-a11a-7d09193e380c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:15.211352 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca6b57-32c0-435e-af19-22f4a4233dc3 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:15.224473 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca6b57-32c0-435e-af19-22f4a4233dc3 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:15.227677 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1411/2843] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:22:15 2022] PUT /identity/v3/projects/85760c10dd0d455ca79e274da45eddaa/users/e9fb9a6d27984900ac2ff05d609e6de2/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:22:15.240243 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab2528-6456-430b-9484-f4023ee9ddbf None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:15.240666 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab2528-6456-430b-9484-f4023ee9ddbf None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:15.240915 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab2528-6456-430b-9484-f4023ee9ddbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:15.275114 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e8ab2528-6456-430b-9484-f4023ee9ddbf None None] MFA Rules not processed for user `e9fb9a6d27984900ac2ff05d609e6de2`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:15.277243 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e8ab2528-6456-430b-9484-f4023ee9ddbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:15.310381 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887d6fe-dfa5-45f8-be1e-052631e26d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:15.311069 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887d6fe-dfa5-45f8-be1e-052631e26d94 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:15.311354 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887d6fe-dfa5-45f8-be1e-052631e26d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:15.311634 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887d6fe-dfa5-45f8-be1e-052631e26d94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:15.312686 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c887d6fe-dfa5-45f8-be1e-052631e26d94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:15.314436 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c887d6fe-dfa5-45f8-be1e-052631e26d94 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:15.355480 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1433/2844] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:22:15 2022] POST /identity/v3/projects => generated 383 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:22:15.357108 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887d6fe-dfa5-45f8-be1e-052631e26d94 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:15.369868 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:15.376554 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1412/2845] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:22:15 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:22:15.377854 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab2528-6456-430b-9484-f4023ee9ddbf None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:15.380671 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e8ab2528-6456-430b-9484-f4023ee9ddbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:15.446543 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887d6fe-dfa5-45f8-be1e-052631e26d94 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:15.447882 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab2528-6456-430b-9484-f4023ee9ddbf None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:15.452083 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f454ca-65c8-42f0-8342-cee6f8466c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:15.452872 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f454ca-65c8-42f0-8342-cee6f8466c23 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:15.453155 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f454ca-65c8-42f0-8342-cee6f8466c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:15.453432 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f454ca-65c8-42f0-8342-cee6f8466c23 admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:15.454101 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f454ca-65c8-42f0-8342-cee6f8466c23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:15.456193 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f454ca-65c8-42f0-8342-cee6f8466c23 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:15.480700 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1434/2846] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:22:15 2022] POST /identity/v3/users => generated 298 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:22:15.484500 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f454ca-65c8-42f0-8342-cee6f8466c23 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:15.485585 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:15.495818 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f454ca-65c8-42f0-8342-cee6f8466c23 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:15.501603 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8af5c3-fedb-468c-b835-5e3336d9fff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:15.502227 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8af5c3-fedb-468c-b835-5e3336d9fff5 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:15.502514 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8af5c3-fedb-468c-b835-5e3336d9fff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:15.502792 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8af5c3-fedb-468c-b835-5e3336d9fff5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:15.503495 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8af5c3-fedb-468c-b835-5e3336d9fff5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:15.505092 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ee584-25bb-4b63-99f1-77255ba1975c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:15.505235 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8af5c3-fedb-468c-b835-5e3336d9fff5 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:15.505719 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ee584-25bb-4b63-99f1-77255ba1975c service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:15.506004 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ee584-25bb-4b63-99f1-77255ba1975c service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:15.506281 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ee584-25bb-4b63-99f1-77255ba1975c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:15.519321 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1435/2847] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 14:22:15.523768 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8af5c3-fedb-468c-b835-5e3336d9fff5 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:15.535749 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8af5c3-fedb-468c-b835-5e3336d9fff5 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:15.541588 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea9974-811f-4f0c-8dec-9c5286554037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:15.542500 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea9974-811f-4f0c-8dec-9c5286554037 admin admin] REQUEST_METHOD: `PUT` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:15.542832 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea9974-811f-4f0c-8dec-9c5286554037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:15.543215 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea9974-811f-4f0c-8dec-9c5286554037 admin admin] PATH_INFO: `/v3/projects/1ecb562719a947008e82c583602c438f/users/5eede8b4bf084829aba44d5e85958576/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:15.564947 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ea9974-811f-4f0c-8dec-9c5286554037 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ecb562719a947008e82c583602c438f, user_id=5eede8b4bf084829aba44d5e85958576, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:15.568359 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ea9974-811f-4f0c-8dec-9c5286554037 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:15.578079 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ee584-25bb-4b63-99f1-77255ba1975c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:15.580239 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ee584-25bb-4b63-99f1-77255ba1975c service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:15.583217 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1436/2848] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:22:15 2022] PUT /identity/v3/projects/1ecb562719a947008e82c583602c438f/users/5eede8b4bf084829aba44d5e85958576/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:22:15.587773 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea9974-811f-4f0c-8dec-9c5286554037 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:15.590893 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1413/2849] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:22:15 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:22:15.681250 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea9974-811f-4f0c-8dec-9c5286554037 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:15.686224 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe74643-c298-40c6-a824-a3b4538ce99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:15.686967 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe74643-c298-40c6-a824-a3b4538ce99a admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:15.687257 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe74643-c298-40c6-a824-a3b4538ce99a admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:15.687540 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe74643-c298-40c6-a824-a3b4538ce99a admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:15.688242 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe74643-c298-40c6-a824-a3b4538ce99a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:15.689937 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe74643-c298-40c6-a824-a3b4538ce99a admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:15.700301 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1437/2850] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:22:15.705137 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ee584-25bb-4b63-99f1-77255ba1975c service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:15.716386 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ee584-25bb-4b63-99f1-77255ba1975c service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:15.721989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a5c4ff-9628-486a-89a3-629b53417b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:15.722667 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a5c4ff-9628-486a-89a3-629b53417b0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:15.722953 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a5c4ff-9628-486a-89a3-629b53417b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:15.723235 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a5c4ff-9628-486a-89a3-629b53417b0a admin admin] PATH_INFO: `/v3/projects/1ecb562719a947008e82c583602c438f/users/5eede8b4bf084829aba44d5e85958576/roles/58f91e3a13664dabb18c6227a1658fa6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:15.725424 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a5c4ff-9628-486a-89a3-629b53417b0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ecb562719a947008e82c583602c438f, user_id=5eede8b4bf084829aba44d5e85958576, role_id=58f91e3a13664dabb18c6227a1658fa6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:15.728120 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a5c4ff-9628-486a-89a3-629b53417b0a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:15.742938 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1414/2851] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:22:15 2022] PUT /identity/v3/projects/1ecb562719a947008e82c583602c438f/users/5eede8b4bf084829aba44d5e85958576/roles/58f91e3a13664dabb18c6227a1658fa6 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:22:15.753975 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de850c7-fcf4-4abc-ad98-d447180e0f49 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:15.754457 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de850c7-fcf4-4abc-ad98-d447180e0f49 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:15.754581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de850c7-fcf4-4abc-ad98-d447180e0f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:15.805572 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-1de850c7-fcf4-4abc-ad98-d447180e0f49 None None] MFA Rules not processed for user `5eede8b4bf084829aba44d5e85958576`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:15.807371 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-1de850c7-fcf4-4abc-ad98-d447180e0f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:15.888595 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:15.925661 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1438/2852] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:22:15 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:22:15.939872 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a5c4ff-9628-486a-89a3-629b53417b0a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:16.031246 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a5c4ff-9628-486a-89a3-629b53417b0a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:16.036177 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4bd9e-65e2-4540-b945-8354ebfc9e22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:16.037061 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4bd9e-65e2-4540-b945-8354ebfc9e22 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:16.037360 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4bd9e-65e2-4540-b945-8354ebfc9e22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:16.037645 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4bd9e-65e2-4540-b945-8354ebfc9e22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:16.098447 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b4bd9e-65e2-4540-b945-8354ebfc9e22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:16.099887 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b4bd9e-65e2-4540-b945-8354ebfc9e22 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:16.108961 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1415/2853] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:22:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:22:17.686702 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de850c7-fcf4-4abc-ad98-d447180e0f49 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:17.688143 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-1de850c7-fcf4-4abc-ad98-d447180e0f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:17.700669 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de850c7-fcf4-4abc-ad98-d447180e0f49 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:17.705224 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f171580d-3f8e-43b6-9a9e-b6ecd897a2d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:17.705846 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171580d-3f8e-43b6-9a9e-b6ecd897a2d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:17.706132 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171580d-3f8e-43b6-9a9e-b6ecd897a2d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:17.706408 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171580d-3f8e-43b6-9a9e-b6ecd897a2d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:17.769328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f171580d-3f8e-43b6-9a9e-b6ecd897a2d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:17.771366 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f171580d-3f8e-43b6-9a9e-b6ecd897a2d6 service cinder] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:17.782440 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1439/2854] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:22:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:22:18.421567 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4bd9e-65e2-4540-b945-8354ebfc9e22 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:18.430940 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4bd9e-65e2-4540-b945-8354ebfc9e22 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:18.435671 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402cd167-2eb6-4322-b1d9-4af7d9fc9e00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:18.436367 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402cd167-2eb6-4322-b1d9-4af7d9fc9e00 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:18.436756 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402cd167-2eb6-4322-b1d9-4af7d9fc9e00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:18.437102 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402cd167-2eb6-4322-b1d9-4af7d9fc9e00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:18.449554 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402cd167-2eb6-4322-b1d9-4af7d9fc9e00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:18.451257 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402cd167-2eb6-4322-b1d9-4af7d9fc9e00 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:18.464032 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1416/2855] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:22:18 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:22:20.166822 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f171580d-3f8e-43b6-9a9e-b6ecd897a2d6 service cinder] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:20.168371 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-f171580d-3f8e-43b6-9a9e-b6ecd897a2d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:20.234097 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f171580d-3f8e-43b6-9a9e-b6ecd897a2d6 service cinder] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:20.239167 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a443d82-8af0-440d-9022-1f8195dad898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:20.239740 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a443d82-8af0-440d-9022-1f8195dad898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:20.240036 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a443d82-8af0-440d-9022-1f8195dad898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:20.240292 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a443d82-8af0-440d-9022-1f8195dad898 admin admin] PATH_INFO: `/v3/users/2eb09df64dd74ddeb9fbbf44ae0a3592` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:20.241489 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a443d82-8af0-440d-9022-1f8195dad898 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2eb09df64dd74ddeb9fbbf44ae0a3592)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:20.243383 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a443d82-8af0-440d-9022-1f8195dad898 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:20.290071 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-6a443d82-8af0-440d-9022-1f8195dad898 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2eb09df64dd74ddeb9fbbf44ae0a3592'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:20.290552 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-6a443d82-8af0-440d-9022-1f8195dad898 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2eb09df64dd74ddeb9fbbf44ae0a3592'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:20.302802 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-6a443d82-8af0-440d-9022-1f8195dad898 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2eb09df64dd74ddeb9fbbf44ae0a3592'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:20.309801 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-6a443d82-8af0-440d-9022-1f8195dad898 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2eb09df64dd74ddeb9fbbf44ae0a3592'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:20.310460 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-6a443d82-8af0-440d-9022-1f8195dad898 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2eb09df64dd74ddeb9fbbf44ae0a3592'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:20.324137 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1440/2856] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:22:20 2022] DELETE /identity/v3/users/2eb09df64dd74ddeb9fbbf44ae0a3592 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:22:20.499401 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402cd167-2eb6-4322-b1d9-4af7d9fc9e00 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:20.510668 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402cd167-2eb6-4322-b1d9-4af7d9fc9e00 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:20.528255 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:20.570508 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0310b770-fbca-41bd-a821-4f4506d449cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:20.571163 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310b770-fbca-41bd-a821-4f4506d449cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:20.571436 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310b770-fbca-41bd-a821-4f4506d449cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:20.571717 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310b770-fbca-41bd-a821-4f4506d449cb admin admin] PATH_INFO: `/v3/projects/7bb93e302f1441fca85409de83230d6a` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:20.572909 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0310b770-fbca-41bd-a821-4f4506d449cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bb93e302f1441fca85409de83230d6a)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:20.574819 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0310b770-fbca-41bd-a821-4f4506d449cb admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:20.621176 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0310b770-fbca-41bd-a821-4f4506d449cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bb93e302f1441fca85409de83230d6a', 'request_id': 'req-0310b770-fbca-41bd-a821-4f4506d449cb'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:20.628773 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0310b770-fbca-41bd-a821-4f4506d449cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bb93e302f1441fca85409de83230d6a', 'request_id': 'req-0310b770-fbca-41bd-a821-4f4506d449cb'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:20.635069 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0310b770-fbca-41bd-a821-4f4506d449cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bb93e302f1441fca85409de83230d6a', 'request_id': 'req-0310b770-fbca-41bd-a821-4f4506d449cb'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:20.645435 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0310b770-fbca-41bd-a821-4f4506d449cb admin admin] The token cache is being invalidate because project 7bb93e302f1441fca85409de83230d6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:22:20.645764 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0310b770-fbca-41bd-a821-4f4506d449cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:20.646407 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0310b770-fbca-41bd-a821-4f4506d449cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:20.647935 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1417/2857] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:22:20 2022] DELETE /identity/v3/projects/7bb93e302f1441fca85409de83230d6a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:22:20.652126 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a443d82-8af0-440d-9022-1f8195dad898 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:20.653596 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-6a443d82-8af0-440d-9022-1f8195dad898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:20.662168 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:20.668941 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a443d82-8af0-440d-9022-1f8195dad898 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:20.727095 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c255b0-f512-44a0-903b-166486dbc152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:20.727742 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c255b0-f512-44a0-903b-166486dbc152 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:20.728029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c255b0-f512-44a0-903b-166486dbc152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:20.728306 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c255b0-f512-44a0-903b-166486dbc152 admin admin] PATH_INFO: `/v3/users/73ece04e4ef041f7a949ecb541b748a6` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:20.743198 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c255b0-f512-44a0-903b-166486dbc152 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73ece04e4ef041f7a949ecb541b748a6)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:20.745146 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c255b0-f512-44a0-903b-166486dbc152 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:20.794657 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d5c255b0-f512-44a0-903b-166486dbc152 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73ece04e4ef041f7a949ecb541b748a6'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:20.795325 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d5c255b0-f512-44a0-903b-166486dbc152 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73ece04e4ef041f7a949ecb541b748a6'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:20.808974 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d5c255b0-f512-44a0-903b-166486dbc152 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73ece04e4ef041f7a949ecb541b748a6'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:20.821392 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d5c255b0-f512-44a0-903b-166486dbc152 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73ece04e4ef041f7a949ecb541b748a6'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:20.822038 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-d5c255b0-f512-44a0-903b-166486dbc152 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73ece04e4ef041f7a949ecb541b748a6'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:20.836017 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1441/2858] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:22:20 2022] DELETE /identity/v3/users/73ece04e4ef041f7a949ecb541b748a6 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:22:21.011968 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0310b770-fbca-41bd-a821-4f4506d449cb admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:21.013423 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0310b770-fbca-41bd-a821-4f4506d449cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:21.028026 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0310b770-fbca-41bd-a821-4f4506d449cb admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:21.099653 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abfcf9d-0363-4fb5-a700-f46740ebe962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:21.100584 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abfcf9d-0363-4fb5-a700-f46740ebe962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:21.100788 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abfcf9d-0363-4fb5-a700-f46740ebe962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:21.101069 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abfcf9d-0363-4fb5-a700-f46740ebe962 admin admin] PATH_INFO: `/v3/projects/36e85992ca424ff6916434fd4f71c92d` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:21.114161 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abfcf9d-0363-4fb5-a700-f46740ebe962 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36e85992ca424ff6916434fd4f71c92d)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:21.116942 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abfcf9d-0363-4fb5-a700-f46740ebe962 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:21.175598 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:21.180743 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5abfcf9d-0363-4fb5-a700-f46740ebe962 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36e85992ca424ff6916434fd4f71c92d', 'request_id': 'req-5abfcf9d-0363-4fb5-a700-f46740ebe962'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:21.191447 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5abfcf9d-0363-4fb5-a700-f46740ebe962 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36e85992ca424ff6916434fd4f71c92d', 'request_id': 'req-5abfcf9d-0363-4fb5-a700-f46740ebe962'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:21.210491 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5abfcf9d-0363-4fb5-a700-f46740ebe962 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36e85992ca424ff6916434fd4f71c92d', 'request_id': 'req-5abfcf9d-0363-4fb5-a700-f46740ebe962'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:21.210974 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5abfcf9d-0363-4fb5-a700-f46740ebe962 admin admin] The token cache is being invalidate because project 36e85992ca424ff6916434fd4f71c92d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:22:21.211264 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5abfcf9d-0363-4fb5-a700-f46740ebe962 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:21.211949 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5abfcf9d-0363-4fb5-a700-f46740ebe962 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:21.224775 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1418/2859] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:22:21 2022] DELETE /identity/v3/projects/36e85992ca424ff6916434fd4f71c92d => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:22:22.581200 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c255b0-f512-44a0-903b-166486dbc152 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:22.582613 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-d5c255b0-f512-44a0-903b-166486dbc152 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:22.596021 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c255b0-f512-44a0-903b-166486dbc152 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:22.654212 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b89eb0-78d0-4229-81af-c74480c39e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:22.654862 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b89eb0-78d0-4229-81af-c74480c39e30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:22.655152 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b89eb0-78d0-4229-81af-c74480c39e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:22.655426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b89eb0-78d0-4229-81af-c74480c39e30 admin admin] PATH_INFO: `/v3/users/b57f5d33ff234d2092b47148eb58e4d4` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:22.656626 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b89eb0-78d0-4229-81af-c74480c39e30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b57f5d33ff234d2092b47148eb58e4d4)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:22.660141 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b89eb0-78d0-4229-81af-c74480c39e30 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:22.675560 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:22.711992 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-66b89eb0-78d0-4229-81af-c74480c39e30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b57f5d33ff234d2092b47148eb58e4d4'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:22.712732 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-66b89eb0-78d0-4229-81af-c74480c39e30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b57f5d33ff234d2092b47148eb58e4d4'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:22.728739 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-66b89eb0-78d0-4229-81af-c74480c39e30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b57f5d33ff234d2092b47148eb58e4d4'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:22.737766 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-66b89eb0-78d0-4229-81af-c74480c39e30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b57f5d33ff234d2092b47148eb58e4d4'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:22.738526 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-66b89eb0-78d0-4229-81af-c74480c39e30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b57f5d33ff234d2092b47148eb58e4d4'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:22.755366 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1442/2860] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:22:22 2022] DELETE /identity/v3/users/b57f5d33ff234d2092b47148eb58e4d4 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:22:22.756997 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d1d5b-f5cd-4947-921d-7bf9b88d1b99 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:22.757447 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d1d5b-f5cd-4947-921d-7bf9b88d1b99 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:22.757541 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d1d5b-f5cd-4947-921d-7bf9b88d1b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:22.789010 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-7d1d1d5b-f5cd-4947-921d-7bf9b88d1b99 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:22.790287 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7d1d1d5b-f5cd-4947-921d-7bf9b88d1b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:22.857434 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1419/2861] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:22:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:22:22.867908 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b89eb0-78d0-4229-81af-c74480c39e30 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:22.869368 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-66b89eb0-78d0-4229-81af-c74480c39e30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:22.925746 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b89eb0-78d0-4229-81af-c74480c39e30 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:22.929343 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9408d0e-e817-4688-bca0-4b0955f0cc01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:22.929957 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9408d0e-e817-4688-bca0-4b0955f0cc01 service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:22.930240 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9408d0e-e817-4688-bca0-4b0955f0cc01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:22.930516 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9408d0e-e817-4688-bca0-4b0955f0cc01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:22.939857 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9408d0e-e817-4688-bca0-4b0955f0cc01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:22.941318 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9408d0e-e817-4688-bca0-4b0955f0cc01 service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:22.951120 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1443/2862] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:22:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:22:23.233879 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d1d5b-f5cd-4947-921d-7bf9b88d1b99 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:23.235422 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7d1d1d5b-f5cd-4947-921d-7bf9b88d1b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:23.248807 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d1d5b-f5cd-4947-921d-7bf9b88d1b99 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:23.255112 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0509d-498d-40ae-b3a7-dbf52011d623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:23.256104 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0509d-498d-40ae-b3a7-dbf52011d623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:23.256595 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0509d-498d-40ae-b3a7-dbf52011d623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:23.257010 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0509d-498d-40ae-b3a7-dbf52011d623 admin admin] PATH_INFO: `/v3/projects/1b86711947064f35bc72e8f352ee33b4` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:23.258706 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d0509d-498d-40ae-b3a7-dbf52011d623 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b86711947064f35bc72e8f352ee33b4)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:23.260709 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d0509d-498d-40ae-b3a7-dbf52011d623 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:23.308184 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d3d0509d-498d-40ae-b3a7-dbf52011d623 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b86711947064f35bc72e8f352ee33b4', 'request_id': 'req-d3d0509d-498d-40ae-b3a7-dbf52011d623'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:23.315352 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d3d0509d-498d-40ae-b3a7-dbf52011d623 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b86711947064f35bc72e8f352ee33b4', 'request_id': 'req-d3d0509d-498d-40ae-b3a7-dbf52011d623'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:23.322225 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d3d0509d-498d-40ae-b3a7-dbf52011d623 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b86711947064f35bc72e8f352ee33b4', 'request_id': 'req-d3d0509d-498d-40ae-b3a7-dbf52011d623'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:23.332869 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d3d0509d-498d-40ae-b3a7-dbf52011d623 admin admin] The token cache is being invalidate because project 1b86711947064f35bc72e8f352ee33b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:22:23.333143 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d3d0509d-498d-40ae-b3a7-dbf52011d623 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:23.333768 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d3d0509d-498d-40ae-b3a7-dbf52011d623 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:23.335265 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1420/2863] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:22:23 2022] DELETE /identity/v3/projects/1b86711947064f35bc72e8f352ee33b4 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:22:23.339416 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9408d0e-e817-4688-bca0-4b0955f0cc01 service neutron] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:23.349759 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9408d0e-e817-4688-bca0-4b0955f0cc01 service neutron] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:23.407057 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e4fef7-5321-4dd8-a4aa-008f5bc0a30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:23.407754 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4fef7-5321-4dd8-a4aa-008f5bc0a30d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:23.408069 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4fef7-5321-4dd8-a4aa-008f5bc0a30d admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:23.408353 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4fef7-5321-4dd8-a4aa-008f5bc0a30d admin admin] PATH_INFO: `/v3/users/b284a1110a1f4179bdd2b6071249ca03` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:23.409579 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e4fef7-5321-4dd8-a4aa-008f5bc0a30d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b284a1110a1f4179bdd2b6071249ca03)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:23.411551 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e4fef7-5321-4dd8-a4aa-008f5bc0a30d admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:23.448184 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:23.459911 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-38e4fef7-5321-4dd8-a4aa-008f5bc0a30d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b284a1110a1f4179bdd2b6071249ca03'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:23.460618 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-38e4fef7-5321-4dd8-a4aa-008f5bc0a30d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b284a1110a1f4179bdd2b6071249ca03'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:23.472663 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-38e4fef7-5321-4dd8-a4aa-008f5bc0a30d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b284a1110a1f4179bdd2b6071249ca03'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:23.480415 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-38e4fef7-5321-4dd8-a4aa-008f5bc0a30d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b284a1110a1f4179bdd2b6071249ca03'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:23.481081 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-38e4fef7-5321-4dd8-a4aa-008f5bc0a30d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b284a1110a1f4179bdd2b6071249ca03'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:23.493306 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1444/2864] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:22:23 2022] DELETE /identity/v3/users/b284a1110a1f4179bdd2b6071249ca03 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:22:23.779529 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0509d-498d-40ae-b3a7-dbf52011d623 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:23.781095 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d3d0509d-498d-40ae-b3a7-dbf52011d623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:23.795583 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0509d-498d-40ae-b3a7-dbf52011d623 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:23.856622 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:23.864654 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ccb229-5d6d-4e00-80d2-c37a97de1068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:23.865344 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ccb229-5d6d-4e00-80d2-c37a97de1068 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:23.865923 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ccb229-5d6d-4e00-80d2-c37a97de1068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:23.866239 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ccb229-5d6d-4e00-80d2-c37a97de1068 admin admin] PATH_INFO: `/v3/projects/aec6c04dc1dd41df827993966bb8c6f0` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:23.867408 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ccb229-5d6d-4e00-80d2-c37a97de1068 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aec6c04dc1dd41df827993966bb8c6f0)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:23.869899 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ccb229-5d6d-4e00-80d2-c37a97de1068 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:23.916809 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-65ccb229-5d6d-4e00-80d2-c37a97de1068 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aec6c04dc1dd41df827993966bb8c6f0', 'request_id': 'req-65ccb229-5d6d-4e00-80d2-c37a97de1068'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:23.924414 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-65ccb229-5d6d-4e00-80d2-c37a97de1068 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aec6c04dc1dd41df827993966bb8c6f0', 'request_id': 'req-65ccb229-5d6d-4e00-80d2-c37a97de1068'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:23.930630 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-65ccb229-5d6d-4e00-80d2-c37a97de1068 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aec6c04dc1dd41df827993966bb8c6f0', 'request_id': 'req-65ccb229-5d6d-4e00-80d2-c37a97de1068'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:23.941189 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-65ccb229-5d6d-4e00-80d2-c37a97de1068 admin admin] The token cache is being invalidate because project aec6c04dc1dd41df827993966bb8c6f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:22:23.941484 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-65ccb229-5d6d-4e00-80d2-c37a97de1068 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:23.942109 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-65ccb229-5d6d-4e00-80d2-c37a97de1068 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:23.944121 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1421/2865] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:22:23 2022] DELETE /identity/v3/projects/aec6c04dc1dd41df827993966bb8c6f0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:22:23.948137 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e4fef7-5321-4dd8-a4aa-008f5bc0a30d admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:23.949552 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-38e4fef7-5321-4dd8-a4aa-008f5bc0a30d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:23.962720 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e4fef7-5321-4dd8-a4aa-008f5bc0a30d admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:24.018298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc22737-e1d8-41ef-aa1b-fea5ce8d62d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:24.018946 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc22737-e1d8-41ef-aa1b-fea5ce8d62d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:24.019231 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc22737-e1d8-41ef-aa1b-fea5ce8d62d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:24.019510 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc22737-e1d8-41ef-aa1b-fea5ce8d62d4 admin admin] PATH_INFO: `/v3/users/5eede8b4bf084829aba44d5e85958576` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:24.020710 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc22737-e1d8-41ef-aa1b-fea5ce8d62d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5eede8b4bf084829aba44d5e85958576)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:24.022729 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc22737-e1d8-41ef-aa1b-fea5ce8d62d4 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:24.068843 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-7cc22737-e1d8-41ef-aa1b-fea5ce8d62d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5eede8b4bf084829aba44d5e85958576'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:24.069498 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-7cc22737-e1d8-41ef-aa1b-fea5ce8d62d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5eede8b4bf084829aba44d5e85958576'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:24.081662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-7cc22737-e1d8-41ef-aa1b-fea5ce8d62d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5eede8b4bf084829aba44d5e85958576'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:24.089016 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-7cc22737-e1d8-41ef-aa1b-fea5ce8d62d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5eede8b4bf084829aba44d5e85958576'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:24.089639 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-7cc22737-e1d8-41ef-aa1b-fea5ce8d62d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5eede8b4bf084829aba44d5e85958576'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:24.102695 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1445/2866] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:22:23 2022] DELETE /identity/v3/users/5eede8b4bf084829aba44d5e85958576 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:22:24.373083 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ccb229-5d6d-4e00-80d2-c37a97de1068 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:24.374682 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-65ccb229-5d6d-4e00-80d2-c37a97de1068 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:24.388956 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ccb229-5d6d-4e00-80d2-c37a97de1068 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:24.449283 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e4be2c-e724-45e8-b24a-adaced5625fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:24.449936 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e4be2c-e724-45e8-b24a-adaced5625fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:24.450243 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e4be2c-e724-45e8-b24a-adaced5625fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:24.450538 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e4be2c-e724-45e8-b24a-adaced5625fe admin admin] PATH_INFO: `/v3/projects/1ecb562719a947008e82c583602c438f` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:24.451830 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e4be2c-e724-45e8-b24a-adaced5625fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ecb562719a947008e82c583602c438f)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:24.453929 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e4be2c-e724-45e8-b24a-adaced5625fe admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:24.503422 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-15e4be2c-e724-45e8-b24a-adaced5625fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ecb562719a947008e82c583602c438f', 'request_id': 'req-15e4be2c-e724-45e8-b24a-adaced5625fe'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:24.510805 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-15e4be2c-e724-45e8-b24a-adaced5625fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ecb562719a947008e82c583602c438f', 'request_id': 'req-15e4be2c-e724-45e8-b24a-adaced5625fe'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:24.517852 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-15e4be2c-e724-45e8-b24a-adaced5625fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ecb562719a947008e82c583602c438f', 'request_id': 'req-15e4be2c-e724-45e8-b24a-adaced5625fe'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:24.528635 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-15e4be2c-e724-45e8-b24a-adaced5625fe admin admin] The token cache is being invalidate because project 1ecb562719a947008e82c583602c438f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:22:24.528925 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-15e4be2c-e724-45e8-b24a-adaced5625fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:24.529578 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-15e4be2c-e724-45e8-b24a-adaced5625fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:24.531114 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1422/2867] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:22:24 2022] DELETE /identity/v3/projects/1ecb562719a947008e82c583602c438f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:22:24.535997 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083edff-dd39-46b6-9813-abc2f96f5004 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:24.536284 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083edff-dd39-46b6-9813-abc2f96f5004 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:24.536588 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083edff-dd39-46b6-9813-abc2f96f5004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:24.556408 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-a083edff-dd39-46b6-9813-abc2f96f5004 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:24.557627 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-a083edff-dd39-46b6-9813-abc2f96f5004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:24.583061 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:24.617567 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1446/2868] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:22:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:22:24.627443 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cef974-eb29-4d15-93c8-7456bc6ddc79 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:24.627672 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cef974-eb29-4d15-93c8-7456bc6ddc79 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:24.627955 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cef974-eb29-4d15-93c8-7456bc6ddc79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:24.648004 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-82cef974-eb29-4d15-93c8-7456bc6ddc79 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:24.649290 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-82cef974-eb29-4d15-93c8-7456bc6ddc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:24.657156 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1423/2869] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:22:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:22:24.661462 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a083edff-dd39-46b6-9813-abc2f96f5004 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:24.671283 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a083edff-dd39-46b6-9813-abc2f96f5004 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:24.676318 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59a6a8-528a-41cc-886e-d04abbb45f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:24.676957 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59a6a8-528a-41cc-886e-d04abbb45f82 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:24.677244 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59a6a8-528a-41cc-886e-d04abbb45f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:24.677551 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59a6a8-528a-41cc-886e-d04abbb45f82 admin admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:24.678249 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c59a6a8-528a-41cc-886e-d04abbb45f82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:22:24.678556 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c59a6a8-528a-41cc-886e-d04abbb45f82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:24.680142 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c59a6a8-528a-41cc-886e-d04abbb45f82 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:24.692662 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1447/2870] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:22:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:22:24.696961 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cef974-eb29-4d15-93c8-7456bc6ddc79 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:24.706426 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cef974-eb29-4d15-93c8-7456bc6ddc79 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:24.711289 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a596a-39c2-427d-b738-2314a50de8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:24.711901 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a596a-39c2-427d-b738-2314a50de8ea admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:24.712225 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a596a-39c2-427d-b738-2314a50de8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:24.712574 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a596a-39c2-427d-b738-2314a50de8ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:24.713305 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a596a-39c2-427d-b738-2314a50de8ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:24.715289 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a596a-39c2-427d-b738-2314a50de8ea admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:24.720742 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:24.745647 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1424/2871] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:22:24 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:22:24.749652 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59a6a8-528a-41cc-886e-d04abbb45f82 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:24.760824 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59a6a8-528a-41cc-886e-d04abbb45f82 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:24.819039 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb89386-51e0-4398-b296-5fb0a8fb18f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:24.819661 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb89386-51e0-4398-b296-5fb0a8fb18f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:24.820445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb89386-51e0-4398-b296-5fb0a8fb18f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:24.820445 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb89386-51e0-4398-b296-5fb0a8fb18f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:24.820910 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb89386-51e0-4398-b296-5fb0a8fb18f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:24.822819 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb89386-51e0-4398-b296-5fb0a8fb18f4 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:24.843323 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1448/2872] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:22:24 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:22:24.848098 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a596a-39c2-427d-b738-2314a50de8ea admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:24.859655 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a596a-39c2-427d-b738-2314a50de8ea admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:24.864912 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39666c1d-fd5a-4307-9de0-1bd1bd863b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:24.865525 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39666c1d-fd5a-4307-9de0-1bd1bd863b8c admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:24.865915 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39666c1d-fd5a-4307-9de0-1bd1bd863b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:24.866139 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39666c1d-fd5a-4307-9de0-1bd1bd863b8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:24.866838 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39666c1d-fd5a-4307-9de0-1bd1bd863b8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:24.868458 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39666c1d-fd5a-4307-9de0-1bd1bd863b8c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:24.879079 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1425/2873] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:22:24.883530 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb89386-51e0-4398-b296-5fb0a8fb18f4 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:24.893801 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb89386-51e0-4398-b296-5fb0a8fb18f4 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:24.898401 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33062918-881c-46c1-82e5-f6ed63ac710e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:24.899043 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33062918-881c-46c1-82e5-f6ed63ac710e admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:24.899330 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33062918-881c-46c1-82e5-f6ed63ac710e admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:24.899609 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33062918-881c-46c1-82e5-f6ed63ac710e admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:24.900307 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33062918-881c-46c1-82e5-f6ed63ac710e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:24.901938 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33062918-881c-46c1-82e5-f6ed63ac710e admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:24.913282 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1449/2874] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:22:24.917606 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39666c1d-fd5a-4307-9de0-1bd1bd863b8c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:24.927556 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39666c1d-fd5a-4307-9de0-1bd1bd863b8c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:24.932395 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcce49b-be0e-43a8-9d65-13e3c238a3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:24.933064 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcce49b-be0e-43a8-9d65-13e3c238a3f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:24.933345 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcce49b-be0e-43a8-9d65-13e3c238a3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:24.933635 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcce49b-be0e-43a8-9d65-13e3c238a3f7 admin admin] PATH_INFO: `/v3/projects/57de109e747f4f80a6c572f194367f6c/users/fedc3dfad8284827855defaedda75283/roles/926b9f36c24847c582c84af904250abc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:24.948219 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcce49b-be0e-43a8-9d65-13e3c238a3f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57de109e747f4f80a6c572f194367f6c, user_id=fedc3dfad8284827855defaedda75283, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:24.950859 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcce49b-be0e-43a8-9d65-13e3c238a3f7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:24.963781 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1426/2875] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:22:24 2022] PUT /identity/v3/projects/57de109e747f4f80a6c572f194367f6c/users/fedc3dfad8284827855defaedda75283/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:22:24.968945 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf7c2f-04c7-486d-a58a-34e04c258be1 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:24.969948 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf7c2f-04c7-486d-a58a-34e04c258be1 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:24.970049 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf7c2f-04c7-486d-a58a-34e04c258be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:24.989668 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-2ecf7c2f-04c7-486d-a58a-34e04c258be1 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:24.990870 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-2ecf7c2f-04c7-486d-a58a-34e04c258be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:25.047944 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1450/2876] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:22:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:22:25.057823 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcce49b-be0e-43a8-9d65-13e3c238a3f7 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:25.059551 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7bcce49b-be0e-43a8-9d65-13e3c238a3f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:25.116921 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcce49b-be0e-43a8-9d65-13e3c238a3f7 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:25.120627 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b9479a-200d-42dc-9a0b-c69ff5efb23c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:25.121307 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b9479a-200d-42dc-9a0b-c69ff5efb23c service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:25.121587 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b9479a-200d-42dc-9a0b-c69ff5efb23c service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:25.121869 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b9479a-200d-42dc-9a0b-c69ff5efb23c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:25.130989 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b9479a-200d-42dc-9a0b-c69ff5efb23c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:25.132359 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b9479a-200d-42dc-9a0b-c69ff5efb23c service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:25.141913 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1427/2877] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:22:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:22:25.759639 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854addc-c4fe-4f4e-9fde-d3653fe8f78c None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:25.759988 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854addc-c4fe-4f4e-9fde-d3653fe8f78c None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:25.760279 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854addc-c4fe-4f4e-9fde-d3653fe8f78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:25.789064 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-3854addc-c4fe-4f4e-9fde-d3653fe8f78c None None] MFA Rules not processed for user `fedc3dfad8284827855defaedda75283`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:25.790304 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-3854addc-c4fe-4f4e-9fde-d3653fe8f78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:25.819131 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:25.861121 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1451/2878] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:22:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:22:25.876005 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b9479a-200d-42dc-9a0b-c69ff5efb23c service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:25.877438 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-22b9479a-200d-42dc-9a0b-c69ff5efb23c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:25.958345 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b9479a-200d-42dc-9a0b-c69ff5efb23c service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:25.962099 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358297af-5a57-4d24-bdd0-cf8511600056 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b185b052f9df4d3f9847d8f9b4cfda5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:25.962747 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358297af-5a57-4d24-bdd0-cf8511600056 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:25.963023 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358297af-5a57-4d24-bdd0-cf8511600056 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:25.963304 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358297af-5a57-4d24-bdd0-cf8511600056 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:25.972240 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358297af-5a57-4d24-bdd0-cf8511600056 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:25.973777 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358297af-5a57-4d24-bdd0-cf8511600056 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:25.982724 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1428/2879] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:22:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:22:28.403195 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15935302-6f97-42c8-8e29-9fa048e2de7f None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:28.403793 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15935302-6f97-42c8-8e29-9fa048e2de7f None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:28.403919 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15935302-6f97-42c8-8e29-9fa048e2de7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:28.424989 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-15935302-6f97-42c8-8e29-9fa048e2de7f None None] MFA Rules not processed for user `fedc3dfad8284827855defaedda75283`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:28.426184 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-15935302-6f97-42c8-8e29-9fa048e2de7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:28.434130 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1452/2880] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:22:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:22:28.699598 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358297af-5a57-4d24-bdd0-cf8511600056 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:28.709502 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358297af-5a57-4d24-bdd0-cf8511600056 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:28.713033 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451300c1-7a6c-4762-ab9a-2abe0a63be05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:28.713677 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451300c1-7a6c-4762-ab9a-2abe0a63be05 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:28.713921 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451300c1-7a6c-4762-ab9a-2abe0a63be05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:28.714197 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451300c1-7a6c-4762-ab9a-2abe0a63be05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:28.715922 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-451300c1-7a6c-4762-ab9a-2abe0a63be05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:28.725205 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451300c1-7a6c-4762-ab9a-2abe0a63be05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:28.726610 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451300c1-7a6c-4762-ab9a-2abe0a63be05 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:28.734762 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1429/2881] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:22:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:22:29.066266 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15935302-6f97-42c8-8e29-9fa048e2de7f None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:29.068077 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-15935302-6f97-42c8-8e29-9fa048e2de7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:29.141219 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15935302-6f97-42c8-8e29-9fa048e2de7f None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:29.146142 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b6c86f-fedd-47e9-a294-b4295f5b425a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e294fe796a1a4ac89041a532856ad627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:29.146750 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6c86f-fedd-47e9-a294-b4295f5b425a service placement] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:29.147028 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6c86f-fedd-47e9-a294-b4295f5b425a service placement] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:29.147301 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6c86f-fedd-47e9-a294-b4295f5b425a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:29.149049 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-35b6c86f-fedd-47e9-a294-b4295f5b425a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:29.160749 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b6c86f-fedd-47e9-a294-b4295f5b425a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:29.162298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b6c86f-fedd-47e9-a294-b4295f5b425a service placement] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:29.171924 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1453/2882] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:22:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:22:37.696069 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451300c1-7a6c-4762-ab9a-2abe0a63be05 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:37.697716 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-451300c1-7a6c-4762-ab9a-2abe0a63be05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:37.751805 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:37.787787 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451300c1-7a6c-4762-ab9a-2abe0a63be05 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:37.793254 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a380b-c96a-4785-9496-f72fe89ff726 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a246654c783b4cc68d212e91bb05d8c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:37.793874 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a380b-c96a-4785-9496-f72fe89ff726 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:37.794158 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a380b-c96a-4785-9496-f72fe89ff726 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:37.794437 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a380b-c96a-4785-9496-f72fe89ff726 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:37.796526 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f16a380b-c96a-4785-9496-f72fe89ff726 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:37.814291 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a380b-c96a-4785-9496-f72fe89ff726 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:37.816393 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a380b-c96a-4785-9496-f72fe89ff726 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:37.828313 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1430/2883] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:22:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:22:39.167860 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=50 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:45.050753 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb9839-022c-4d76-9e22-fad689502a70 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:45.051311 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb9839-022c-4d76-9e22-fad689502a70 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:45.051311 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb9839-022c-4d76-9e22-fad689502a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:45.070780 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-5ccb9839-022c-4d76-9e22-fad689502a70 None None] MFA Rules not processed for user `fedc3dfad8284827855defaedda75283`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:45.071978 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-5ccb9839-022c-4d76-9e22-fad689502a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:45.078776 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1454/2884] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:22:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:22:47.823233 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=14 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:55.065513 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=3 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:22:58.469514 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a380b-c96a-4785-9496-f72fe89ff726 service nova] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:58.470953 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f16a380b-c96a-4785-9496-f72fe89ff726 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:58.486254 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a380b-c96a-4785-9496-f72fe89ff726 service nova] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:58.490694 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dabd8f4-ceb0-490d-a5e5-75c97a0e0363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:58.491325 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dabd8f4-ceb0-490d-a5e5-75c97a0e0363 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:58.491606 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dabd8f4-ceb0-490d-a5e5-75c97a0e0363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:58.491881 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dabd8f4-ceb0-490d-a5e5-75c97a0e0363 admin admin] PATH_INFO: `/v3/users/e9fb9a6d27984900ac2ff05d609e6de2` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:58.493062 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dabd8f4-ceb0-490d-a5e5-75c97a0e0363 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9fb9a6d27984900ac2ff05d609e6de2)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:58.494978 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dabd8f4-ceb0-490d-a5e5-75c97a0e0363 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:58.541112 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7dabd8f4-ceb0-490d-a5e5-75c97a0e0363 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9fb9a6d27984900ac2ff05d609e6de2'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:58.548211 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7dabd8f4-ceb0-490d-a5e5-75c97a0e0363 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9fb9a6d27984900ac2ff05d609e6de2'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:58.548879 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7dabd8f4-ceb0-490d-a5e5-75c97a0e0363 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9fb9a6d27984900ac2ff05d609e6de2'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:58.556318 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7dabd8f4-ceb0-490d-a5e5-75c97a0e0363 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9fb9a6d27984900ac2ff05d609e6de2'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:58.556966 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7dabd8f4-ceb0-490d-a5e5-75c97a0e0363 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9fb9a6d27984900ac2ff05d609e6de2'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:58.573663 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1431/2885] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:22:58 2022] DELETE /identity/v3/users/e9fb9a6d27984900ac2ff05d609e6de2 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:22:58.579272 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6996c-72f1-40f5-a7d9-540006a51774 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:58.579768 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6996c-72f1-40f5-a7d9-540006a51774 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:58.579885 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6996c-72f1-40f5-a7d9-540006a51774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:58.599306 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-47c6996c-72f1-40f5-a7d9-540006a51774 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:22:58.600571 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-47c6996c-72f1-40f5-a7d9-540006a51774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:58.665181 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1455/2886] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:22:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:22:58.675012 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dabd8f4-ceb0-490d-a5e5-75c97a0e0363 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:58.676447 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7dabd8f4-ceb0-490d-a5e5-75c97a0e0363 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:58.735348 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dabd8f4-ceb0-490d-a5e5-75c97a0e0363 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:58.738996 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd1ca2-c1ff-479c-b5a0-ae0f26543a60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:58.739612 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd1ca2-c1ff-479c-b5a0-ae0f26543a60 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:58.739897 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd1ca2-c1ff-479c-b5a0-ae0f26543a60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:58.740176 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd1ca2-c1ff-479c-b5a0-ae0f26543a60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:58.749800 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd1ca2-c1ff-479c-b5a0-ae0f26543a60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:58.751207 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd1ca2-c1ff-479c-b5a0-ae0f26543a60 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:58.761075 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1432/2887] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:22:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:22:59.048314 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c6996c-72f1-40f5-a7d9-540006a51774 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:22:59.049883 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-47c6996c-72f1-40f5-a7d9-540006a51774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:22:59.063117 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c6996c-72f1-40f5-a7d9-540006a51774 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:22:59.067906 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a08f86-0ec1-4a55-af8f-9cc4e82afcac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:22:59.068591 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a08f86-0ec1-4a55-af8f-9cc4e82afcac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:22:59.069011 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a08f86-0ec1-4a55-af8f-9cc4e82afcac admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:22:59.069328 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a08f86-0ec1-4a55-af8f-9cc4e82afcac admin admin] PATH_INFO: `/v3/projects/85760c10dd0d455ca79e274da45eddaa` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:22:59.070631 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a08f86-0ec1-4a55-af8f-9cc4e82afcac admin admin] RBAC: Authorizing `identity:delete_project(project_id=85760c10dd0d455ca79e274da45eddaa)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:22:59.072871 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a08f86-0ec1-4a55-af8f-9cc4e82afcac admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:22:59.124441 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-92a08f86-0ec1-4a55-af8f-9cc4e82afcac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85760c10dd0d455ca79e274da45eddaa', 'request_id': 'req-92a08f86-0ec1-4a55-af8f-9cc4e82afcac'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:59.130655 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-92a08f86-0ec1-4a55-af8f-9cc4e82afcac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85760c10dd0d455ca79e274da45eddaa', 'request_id': 'req-92a08f86-0ec1-4a55-af8f-9cc4e82afcac'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:59.136322 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-92a08f86-0ec1-4a55-af8f-9cc4e82afcac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85760c10dd0d455ca79e274da45eddaa', 'request_id': 'req-92a08f86-0ec1-4a55-af8f-9cc4e82afcac'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:59.149135 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-92a08f86-0ec1-4a55-af8f-9cc4e82afcac admin admin] The token cache is being invalidate because project 85760c10dd0d455ca79e274da45eddaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80140) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:22:59.149449 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-92a08f86-0ec1-4a55-af8f-9cc4e82afcac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:59.150067 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-92a08f86-0ec1-4a55-af8f-9cc4e82afcac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:22:59.151570 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1456/2888] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:22:59 2022] DELETE /identity/v3/projects/85760c10dd0d455ca79e274da45eddaa => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:23:07.173731 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd1ca2-c1ff-479c-b5a0-ae0f26543a60 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:07.175120 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-72fd1ca2-c1ff-479c-b5a0-ae0f26543a60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:07.243644 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd1ca2-c1ff-479c-b5a0-ae0f26543a60 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:07.257287 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:23:07.314727 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a2f0c-c56f-493a-b23a-2956b3036890 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e294fe796a1a4ac89041a532856ad627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:07.315359 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a2f0c-c56f-493a-b23a-2956b3036890 service placement] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:07.315624 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a2f0c-c56f-493a-b23a-2956b3036890 service placement] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:07.316008 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a2f0c-c56f-493a-b23a-2956b3036890 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:07.317877 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8a3a2f0c-c56f-493a-b23a-2956b3036890 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:07.321079 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183b885-711f-4c44-bb97-b04548c90192 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:07.321578 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183b885-711f-4c44-bb97-b04548c90192 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:07.321773 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183b885-711f-4c44-bb97-b04548c90192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:07.329295 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3a2f0c-c56f-493a-b23a-2956b3036890 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:07.330876 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3a2f0c-c56f-493a-b23a-2956b3036890 service placement] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:07.341020 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1433/2889] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:23:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:23:07.341396 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-5183b885-711f-4c44-bb97-b04548c90192 None None] MFA Rules not processed for user `5007a2f8e6244b0788f0a554cc572fe5`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:23:07.342756 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-5183b885-711f-4c44-bb97-b04548c90192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:07.400049 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1457/2890] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 14:23:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 14:23:07.450202 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a2f0c-c56f-493a-b23a-2956b3036890 service placement] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:07.451609 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8a3a2f0c-c56f-493a-b23a-2956b3036890 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:07.539338 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a2f0c-c56f-493a-b23a-2956b3036890 service placement] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:07.545116 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cfa909-d9d7-4a7b-8efe-324a2c2f66d2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2a6d7bd10435ba2c8e552d808b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:07.545739 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cfa909-d9d7-4a7b-8efe-324a2c2f66d2 service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:07.546017 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cfa909-d9d7-4a7b-8efe-324a2c2f66d2 service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:07.546291 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cfa909-d9d7-4a7b-8efe-324a2c2f66d2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:07.555252 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cfa909-d9d7-4a7b-8efe-324a2c2f66d2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:07.556889 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cfa909-d9d7-4a7b-8efe-324a2c2f66d2 service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:07.566062 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1434/2891] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 14:23:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:23:17.392936 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:23:17.562557 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=61 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:23:21.996593 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1a45c-5d31-46c0-a20c-c9d584578118 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:21.997122 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1a45c-5d31-46c0-a20c-c9d584578118 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:21.997122 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1a45c-5d31-46c0-a20c-c9d584578118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:22.016722 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-72c1a45c-5d31-46c0-a20c-c9d584578118 None None] MFA Rules not processed for user `5007a2f8e6244b0788f0a554cc572fe5`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:23:22.017929 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-72c1a45c-5d31-46c0-a20c-c9d584578118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:22.025229 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1458/2892] 10.222.0.48 () {58 vars in 967 bytes} [Mon Aug 1 14:23:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:23:22.037159 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cfa909-d9d7-4a7b-8efe-324a2c2f66d2 service swift] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:22.047572 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cfa909-d9d7-4a7b-8efe-324a2c2f66d2 service swift] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:22.052465 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65cd08b-d1dc-484d-a60a-e2513d8615c3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2a6d7bd10435ba2c8e552d808b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:22.053088 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cd08b-d1dc-484d-a60a-e2513d8615c3 service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:22.053624 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cd08b-d1dc-484d-a60a-e2513d8615c3 service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:22.053896 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cd08b-d1dc-484d-a60a-e2513d8615c3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:22.062285 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65cd08b-d1dc-484d-a60a-e2513d8615c3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:22.063843 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65cd08b-d1dc-484d-a60a-e2513d8615c3 service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:22.071871 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1435/2893] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 14:23:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:23:23.984495 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1a45c-5d31-46c0-a20c-c9d584578118 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:23.986094 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-72c1a45c-5d31-46c0-a20c-c9d584578118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:24.050581 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1a45c-5d31-46c0-a20c-c9d584578118 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:24.055345 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946b772-4c20-4f22-87d3-92c7f2cf616e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:24.055989 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946b772-4c20-4f22-87d3-92c7f2cf616e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:24.056272 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946b772-4c20-4f22-87d3-92c7f2cf616e admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:24.056582 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946b772-4c20-4f22-87d3-92c7f2cf616e admin admin] PATH_INFO: `/v3/users/fedc3dfad8284827855defaedda75283` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:24.057768 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e946b772-4c20-4f22-87d3-92c7f2cf616e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fedc3dfad8284827855defaedda75283)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:24.060274 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e946b772-4c20-4f22-87d3-92c7f2cf616e admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:24.110426 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-e946b772-4c20-4f22-87d3-92c7f2cf616e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fedc3dfad8284827855defaedda75283'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:24.111124 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-e946b772-4c20-4f22-87d3-92c7f2cf616e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fedc3dfad8284827855defaedda75283'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:24.125372 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-e946b772-4c20-4f22-87d3-92c7f2cf616e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fedc3dfad8284827855defaedda75283'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:24.133799 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-e946b772-4c20-4f22-87d3-92c7f2cf616e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fedc3dfad8284827855defaedda75283'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:24.134457 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-e946b772-4c20-4f22-87d3-92c7f2cf616e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fedc3dfad8284827855defaedda75283'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:24.150225 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1459/2894] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:23:23 2022] DELETE /identity/v3/users/fedc3dfad8284827855defaedda75283 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:23:24.333851 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65cd08b-d1dc-484d-a60a-e2513d8615c3 service swift] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:24.347407 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65cd08b-d1dc-484d-a60a-e2513d8615c3 service swift] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:24.412279 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd7d38-f933-4ba0-acfe-91585d1a1473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:24.412958 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd7d38-f933-4ba0-acfe-91585d1a1473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:24.413245 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd7d38-f933-4ba0-acfe-91585d1a1473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:24.413525 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd7d38-f933-4ba0-acfe-91585d1a1473 admin admin] PATH_INFO: `/v3/projects/57de109e747f4f80a6c572f194367f6c` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:24.414671 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd7d38-f933-4ba0-acfe-91585d1a1473 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57de109e747f4f80a6c572f194367f6c)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:24.416632 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd7d38-f933-4ba0-acfe-91585d1a1473 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:24.462429 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6abd7d38-f933-4ba0-acfe-91585d1a1473 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57de109e747f4f80a6c572f194367f6c', 'request_id': 'req-6abd7d38-f933-4ba0-acfe-91585d1a1473'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:24.470174 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6abd7d38-f933-4ba0-acfe-91585d1a1473 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57de109e747f4f80a6c572f194367f6c', 'request_id': 'req-6abd7d38-f933-4ba0-acfe-91585d1a1473'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:24.476886 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6abd7d38-f933-4ba0-acfe-91585d1a1473 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57de109e747f4f80a6c572f194367f6c', 'request_id': 'req-6abd7d38-f933-4ba0-acfe-91585d1a1473'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:24.490079 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6abd7d38-f933-4ba0-acfe-91585d1a1473 admin admin] The token cache is being invalidate because project 57de109e747f4f80a6c572f194367f6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:23:24.490377 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6abd7d38-f933-4ba0-acfe-91585d1a1473 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:24.491064 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6abd7d38-f933-4ba0-acfe-91585d1a1473 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:24.492650 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1436/2895] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:23:24 2022] DELETE /identity/v3/projects/57de109e747f4f80a6c572f194367f6c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:23:24.497527 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72282d7c-73cd-4280-843e-a5f48306853a None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:24.497824 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72282d7c-73cd-4280-843e-a5f48306853a None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:24.498109 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72282d7c-73cd-4280-843e-a5f48306853a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:24.517722 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-72282d7c-73cd-4280-843e-a5f48306853a None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:23:24.518931 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-72282d7c-73cd-4280-843e-a5f48306853a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:24.578659 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1460/2896] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:23:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:23:24.586255 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a65ec4-db9b-490d-8bc8-9748bb2ba6d9 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:24.586567 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a65ec4-db9b-490d-8bc8-9748bb2ba6d9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:24.586854 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a65ec4-db9b-490d-8bc8-9748bb2ba6d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:24.606917 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e8a65ec4-db9b-490d-8bc8-9748bb2ba6d9 None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:23:24.608122 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e8a65ec4-db9b-490d-8bc8-9748bb2ba6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:24.616189 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1437/2897] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:23:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:23:24.620468 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72282d7c-73cd-4280-843e-a5f48306853a None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:24.630836 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72282d7c-73cd-4280-843e-a5f48306853a None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:24.635535 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd9e0b-9a60-4746-9166-55bc54e15d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:24.636740 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd9e0b-9a60-4746-9166-55bc54e15d29 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:24.637029 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd9e0b-9a60-4746-9166-55bc54e15d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:24.637310 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd9e0b-9a60-4746-9166-55bc54e15d29 admin admin] PATH_INFO: `/v3/domains` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:24.638033 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfd9e0b-9a60-4746-9166-55bc54e15d29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80140) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:23:24.638343 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfd9e0b-9a60-4746-9166-55bc54e15d29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:24.639975 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfd9e0b-9a60-4746-9166-55bc54e15d29 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:24.651488 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:23:24.659290 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1461/2898] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 14:23:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:23:24.664964 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a65ec4-db9b-490d-8bc8-9748bb2ba6d9 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:24.674269 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a65ec4-db9b-490d-8bc8-9748bb2ba6d9 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:24.679599 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83232516-7e69-4bdc-952d-aa836fc90e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:24.680430 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83232516-7e69-4bdc-952d-aa836fc90e5e admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:24.680741 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83232516-7e69-4bdc-952d-aa836fc90e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:24.681022 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83232516-7e69-4bdc-952d-aa836fc90e5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:24.681769 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83232516-7e69-4bdc-952d-aa836fc90e5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:24.683747 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83232516-7e69-4bdc-952d-aa836fc90e5e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:24.711680 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1438/2899] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 14:23:24 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:23:24.717511 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd9e0b-9a60-4746-9166-55bc54e15d29 admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:24.728142 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd9e0b-9a60-4746-9166-55bc54e15d29 admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:24.786497 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b71a0-ed60-442a-a592-5c1c59129c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:24.787126 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b71a0-ed60-442a-a592-5c1c59129c8f admin admin] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:24.787409 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b71a0-ed60-442a-a592-5c1c59129c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:24.787685 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b71a0-ed60-442a-a592-5c1c59129c8f admin admin] PATH_INFO: `/v3/users` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:24.788349 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40b71a0-ed60-442a-a592-5c1c59129c8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:24.790298 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40b71a0-ed60-442a-a592-5c1c59129c8f admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:24.813374 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1462/2900] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 14:23:24 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:23:24.817535 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83232516-7e69-4bdc-952d-aa836fc90e5e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:24.828898 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83232516-7e69-4bdc-952d-aa836fc90e5e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:24.833597 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80459b9-e880-4808-9faa-32b2b5956763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:24.834222 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80459b9-e880-4808-9faa-32b2b5956763 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:24.834505 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80459b9-e880-4808-9faa-32b2b5956763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:24.834782 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80459b9-e880-4808-9faa-32b2b5956763 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:24.835483 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80459b9-e880-4808-9faa-32b2b5956763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:24.837132 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80459b9-e880-4808-9faa-32b2b5956763 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:24.848107 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1439/2901] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:23:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:23:24.852459 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b71a0-ed60-442a-a592-5c1c59129c8f admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:24.862597 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b71a0-ed60-442a-a592-5c1c59129c8f admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:24.867447 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fa18ae-8d92-4b4d-98db-270d68767294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:24.868069 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa18ae-8d92-4b4d-98db-270d68767294 admin admin] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:24.868348 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa18ae-8d92-4b4d-98db-270d68767294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:24.868652 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa18ae-8d92-4b4d-98db-270d68767294 admin admin] PATH_INFO: `/v3/roles` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:24.869350 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fa18ae-8d92-4b4d-98db-270d68767294 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:24.871421 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fa18ae-8d92-4b4d-98db-270d68767294 admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:24.882319 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1463/2902] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 14:23:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 14:23:24.887181 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80459b9-e880-4808-9faa-32b2b5956763 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:24.897262 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80459b9-e880-4808-9faa-32b2b5956763 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:24.902081 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11da427c-539d-4def-a816-4e72635fd38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:24.902739 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da427c-539d-4def-a816-4e72635fd38c admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:24.903025 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da427c-539d-4def-a816-4e72635fd38c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:24.903305 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da427c-539d-4def-a816-4e72635fd38c admin admin] PATH_INFO: `/v3/projects/33d304fa4a3147a2ad1a07eea8659885/users/4f651d7176ae4b919aea0530d6cc285e/roles/926b9f36c24847c582c84af904250abc` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:24.918301 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11da427c-539d-4def-a816-4e72635fd38c admin admin] RBAC: Authorizing `identity:create_grant(project_id=33d304fa4a3147a2ad1a07eea8659885, user_id=4f651d7176ae4b919aea0530d6cc285e, role_id=926b9f36c24847c582c84af904250abc)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:24.921062 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11da427c-539d-4def-a816-4e72635fd38c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:24.935754 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1440/2903] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 14:23:24 2022] PUT /identity/v3/projects/33d304fa4a3147a2ad1a07eea8659885/users/4f651d7176ae4b919aea0530d6cc285e/roles/926b9f36c24847c582c84af904250abc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:23:24.945771 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3631d-e933-4bd3-acda-10da50cedbb1 None None] REQUEST_METHOD: `POST` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:24.946065 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3631d-e933-4bd3-acda-10da50cedbb1 None None] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:24.946344 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3631d-e933-4bd3-acda-10da50cedbb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:24.973817 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.auth.core [None req-c8a3631d-e933-4bd3-acda-10da50cedbb1 None None] MFA Rules not processed for user `4f651d7176ae4b919aea0530d6cc285e`. Rule list: `[]` (Enabled: `True`). {{(pid=80140) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:23:24.974962 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-c8a3631d-e933-4bd3-acda-10da50cedbb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:25.040444 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1464/2904] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:23:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:23:25.051928 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11da427c-539d-4def-a816-4e72635fd38c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:25.053903 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-11da427c-539d-4def-a816-4e72635fd38c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:25.071983 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:23:25.134794 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11da427c-539d-4def-a816-4e72635fd38c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:25.139063 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f42df6-5b39-4be8-9aa5-dbe6b00d3e72 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39a66f7362964389be039937a786bb0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:25.139667 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f42df6-5b39-4be8-9aa5-dbe6b00d3e72 service glance] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:25.139946 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f42df6-5b39-4be8-9aa5-dbe6b00d3e72 service glance] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:25.140217 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f42df6-5b39-4be8-9aa5-dbe6b00d3e72 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:25.148418 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f42df6-5b39-4be8-9aa5-dbe6b00d3e72 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:25.149854 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f42df6-5b39-4be8-9aa5-dbe6b00d3e72 service glance] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:25.158587 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1441/2905] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:23:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:23:35.037407 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=66,keystone:INSERT=3 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:23:35.154612 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=29 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:23:57.123000 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a3631d-e933-4bd3-acda-10da50cedbb1 None None] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:57.190250 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a3631d-e933-4bd3-acda-10da50cedbb1 None None] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:57.195009 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe8732-872b-4fab-bcde-3db0f4c16a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:57.195681 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe8732-872b-4fab-bcde-3db0f4c16a5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:57.196554 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe8732-872b-4fab-bcde-3db0f4c16a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:57.196841 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe8732-872b-4fab-bcde-3db0f4c16a5d admin admin] PATH_INFO: `/v3/users/4f651d7176ae4b919aea0530d6cc285e` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:57.198048 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afe8732-872b-4fab-bcde-3db0f4c16a5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f651d7176ae4b919aea0530d6cc285e)` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:57.199995 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afe8732-872b-4fab-bcde-3db0f4c16a5d admin admin] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:57.248540 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-8afe8732-872b-4fab-bcde-3db0f4c16a5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f651d7176ae4b919aea0530d6cc285e'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:57.249236 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-8afe8732-872b-4fab-bcde-3db0f4c16a5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f651d7176ae4b919aea0530d6cc285e'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:57.262041 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-8afe8732-872b-4fab-bcde-3db0f4c16a5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f651d7176ae4b919aea0530d6cc285e'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:57.269938 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-8afe8732-872b-4fab-bcde-3db0f4c16a5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f651d7176ae4b919aea0530d6cc285e'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:57.270583 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.notifications [None req-8afe8732-872b-4fab-bcde-3db0f4c16a5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f651d7176ae4b919aea0530d6cc285e'} {{(pid=80140) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:57.282868 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1465/2906] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 14:23:57 2022] DELETE /identity/v3/users/4f651d7176ae4b919aea0530d6cc285e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:23:57.287681 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced535d-82ff-44d4-8cb4-63bb0761fa0d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:57.288178 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced535d-82ff-44d4-8cb4-63bb0761fa0d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:57.288294 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced535d-82ff-44d4-8cb4-63bb0761fa0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:57.308680 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-7ced535d-82ff-44d4-8cb4-63bb0761fa0d None None] MFA Rules not processed for user `7d5a51326470403490a7155c74035d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:23:57.309907 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7ced535d-82ff-44d4-8cb4-63bb0761fa0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:57.369374 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1442/2907] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 14:23:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 14:23:57.379099 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe8732-872b-4fab-bcde-3db0f4c16a5d admin admin] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:57.380583 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-8afe8732-872b-4fab-bcde-3db0f4c16a5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:57.438884 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe8732-872b-4fab-bcde-3db0f4c16a5d admin admin] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:57.442518 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8e0cc-92ec-4614-b3ea-50baabc5f14e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb9f698e09d246bcbea855613764fbe1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:57.443130 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8e0cc-92ec-4614-b3ea-50baabc5f14e service neutron] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:57.443412 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8e0cc-92ec-4614-b3ea-50baabc5f14e service neutron] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:57.443687 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8e0cc-92ec-4614-b3ea-50baabc5f14e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:57.453503 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8e0cc-92ec-4614-b3ea-50baabc5f14e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:57.454922 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8e0cc-92ec-4614-b3ea-50baabc5f14e service neutron] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:57.464364 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1466/2908] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:23:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 14:23:57.728765 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced535d-82ff-44d4-8cb4-63bb0761fa0d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:23:57.730240 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7ced535d-82ff-44d4-8cb4-63bb0761fa0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:23:57.742441 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced535d-82ff-44d4-8cb4-63bb0761fa0d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:23:57.747660 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1525f7-407f-4248-8f6e-c7ecec7275a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d5a51326470403490a7155c74035d21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b30b7ebe5b2d4e7c8f161e5b78a7a0df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:23:57.748317 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1525f7-407f-4248-8f6e-c7ecec7275a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:23:57.748640 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1525f7-407f-4248-8f6e-c7ecec7275a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:23:57.748940 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1525f7-407f-4248-8f6e-c7ecec7275a3 admin admin] PATH_INFO: `/v3/projects/33d304fa4a3147a2ad1a07eea8659885` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:23:57.750133 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1525f7-407f-4248-8f6e-c7ecec7275a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33d304fa4a3147a2ad1a07eea8659885)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:23:57.752059 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1525f7-407f-4248-8f6e-c7ecec7275a3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:23:57.799712 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9a1525f7-407f-4248-8f6e-c7ecec7275a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33d304fa4a3147a2ad1a07eea8659885', 'request_id': 'req-9a1525f7-407f-4248-8f6e-c7ecec7275a3'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:57.808845 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9a1525f7-407f-4248-8f6e-c7ecec7275a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33d304fa4a3147a2ad1a07eea8659885', 'request_id': 'req-9a1525f7-407f-4248-8f6e-c7ecec7275a3'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:57.815506 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9a1525f7-407f-4248-8f6e-c7ecec7275a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33d304fa4a3147a2ad1a07eea8659885', 'request_id': 'req-9a1525f7-407f-4248-8f6e-c7ecec7275a3'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:57.826889 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9a1525f7-407f-4248-8f6e-c7ecec7275a3 admin admin] The token cache is being invalidate because project 33d304fa4a3147a2ad1a07eea8659885 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 14:23:57.827189 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9a1525f7-407f-4248-8f6e-c7ecec7275a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:57.827848 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9a1525f7-407f-4248-8f6e-c7ecec7275a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 14:23:57.829625 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1443/2909] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 14:23:57 2022] DELETE /identity/v3/projects/33d304fa4a3147a2ad1a07eea8659885 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:24:07.460088 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:24:07.823714 dv-cinder-847131-3-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:28:12.075706 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8e0cc-92ec-4614-b3ea-50baabc5f14e service neutron] Authenticating user token {{(pid=80140) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:28:12.077146 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-31e8e0cc-92ec-4614-b3ea-50baabc5f14e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:12.218568 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8e0cc-92ec-4614-b3ea-50baabc5f14e service neutron] Validating token access rules against request {{(pid=80140) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:28:12.277415 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9641fb4-fd04-4248-abd6-5df3f3b1fed5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e294fe796a1a4ac89041a532856ad627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '72ecec9fad964b1fbeaefd0519597229', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80140) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:28:12.278062 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9641fb4-fd04-4248-abd6-5df3f3b1fed5 service placement] REQUEST_METHOD: `GET` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:28:12.278377 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9641fb4-fd04-4248-abd6-5df3f3b1fed5 service placement] SCRIPT_NAME: `/identity` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:28:12.278660 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9641fb4-fd04-4248-abd6-5df3f3b1fed5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80140) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:28:12.280450 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.fernet_utils [None req-c9641fb4-fd04-4248-abd6-5df3f3b1fed5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80140) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:28:12.292053 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9641fb4-fd04-4248-abd6-5df3f3b1fed5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:28:12.293701 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9641fb4-fd04-4248-abd6-5df3f3b1fed5 service placement] RBAC: Authorization granted {{(pid=80140) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:28:12.304143 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: [pid: 80140|app: 0|req: 1467/2910] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 14:28:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 14:28:22.299368 dv-cinder-847131-3-iscsi devstack@keystone.service[80140]: DEBUG dbcounter [-] [80140] Writing DB stats keystone:SELECT=89 {{(pid=80140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}